CFS2 Mesh

[rhumbaflappy]

In truth, there is no worldwide free mesh of good quality available for CFS2. FSX already has a good base mesh... but for the other Microsoft sims, a process to make decent mesh up to LOD9 levels can seem a mystery.

;----------------------------------------------------------------------------------

Tools needed:

TMFViewer ( FSX' version is the best ):
From the FSX SDK...sorry no free download available.

LWMViewer ( use if no FSX TMFViewer ):
http://www.jimkeir.co.uk/FlightSim/LWMViewer.html

LEC:
http://webpages.charter.net/ludowr/LEC.zip


FS2000 resample, mfcompress, and tmf2bgl:
http://files.fsnordic.net/?download=390
or
included with the above LEC download.

;----------------------------------------------------------------------------------

Mesh starts with the DEM data. DEM = Digital Elevational Model.

For worldwide mesh development, there is one adequate source. This is the CGIAR SRTM-based v4 DEMs. http://srtm.csi.cgiar.org/index.asp

The download area is here: http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp

The data covers from N60* to S60*, and is hole-filled and noise reduced. There may be better regional mesh available for free, but for the worlwide SRTM set, this is it.

The data at CGIAR may be identified by inputting coordinates. What will be returned is a group of files to download. What is the size to input? What size do you wish to make?

I use LEC.exe to calculate my extents. LOD2-size corresponds to the cfs2 "CST" file size, and can work for LOD 5-->7 mesh resolution... LOD5-size works best for LOD 8-->9 mesh resolution. Above LOD 9 resolution, you need a better regional DEM dataset.

Start small. LOD5 sized is the default of FS9 and FSX folders. You can later develop LWM and VTP vectors for land/water masks and streams. Concentrate on areas you wish to fly or fight. Use common FS9 names, or CFS2 Regional names. Using LEC, you can find the DEM bounds. I use these bounds for the DEM search... but I'll add a degree to pad the data! You ALWAYS need source data larger than output data.

An LOD 5-sized mesh could use N44,W89 as a reference point in LEC. Generating the bounds with exact extents yields:

North 45.0
West -90.0
South 42.1875
East -86.25

I change this to the following for my search:

North 46.0 Latitude MAX
West -91.0 Longitude MIN
South 41.1875 Latitude MIN
East -85.25 Longitude MAX

;----------------------------------------------------------------------------------

...and this is what you need to input into

http://srtm.csi.cgiar.org/SELECTION/inputCoord.asp

as decimal degrees input. Make sure to select GeoTiff as the type of data, and input coordinates.

For this example I get 4 sourcefiles as selections. I don't trust any of the mirrror sites, so write down the needed zip names, and go to:

ftp://srtm.csi.cgiar.org/SRTM_v41/SRTM_Data_GeoTIFF/

and download the needed files from there. They will be the latest ( v41 )

Download all to a desktop folder named "MyMesh". Into "MyMesh" place resample, tmfcompress and tmf2bgl. UnZip the files: srtm_18_04.zip, srtm_19_04.zip, srtm_18_03.zip, srtm_19_03.zip

We're done with CGIAR.

;----------------------------------------------------------------------------------

Using MicroDem File/Open/OpenDEM, open each tif file in turn.

srtm_19_03 has missing data in lake Superior... 3Dem shows this as an odd square of data as missing instead of 179 meters... We need to fill this as 179 meters before saving as BIL: Edit/FlattenLake... select the color ( probably black ). LakeLevel 179. Some DEMs may have ocean as missing data... Edit/DEMHoles/MissingDataToSeaLevel.

Then, File/SaveDEM/BIL. Make sure you cahnge the save directory each time to your MyMesh folder.

;----------------------------------------------------------------------------------

We now have 4 BIL files. We also have 4 BLW and HDR files.. The BLW and HDR files are text files, and we may need their data for making our INF file for resample.

Now, create a new text file and save it as the FS9 name from LEC: 924160m.INF ( the m will stand for mesh ), and save it to the MyMesh folder:

;----------------------------------------------------------------------------------
[Destination]
; the lod could be 5, 6, 7, 8, or 9
LOD = 9
DestDir = "."
DestBaseFileName = "924160m"
UseSourceDimensions = 0
; exact extents from LEC
NorthLat = 45
SouthLat = 42.1875
WestLong = -90
EastLong = -86.25

; 4 16 bit binary files from microdem

[Source]
Type = MultiSource
NumberOfSources = 4

[Source1]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "SRTM_18_03.BIL"
Lat = 50.00000000000000
Lon = -95.00000000000000
NumOfLines = 6000
NumOfCellsPerLine = 6000
CellXdimensionDeg = 0.00083333335351
CellYdimensionDeg = 0.00083333335351
ScaleinMeters = 1.0

[Source2]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "SRTM_18_04.BIL"
Lat = 45.00000000000000
Lon = -95.00000000000000
NumOfLines = 6000
NumOfCellsPerLine = 6000
CellXdimensionDeg = 0.00083333335351
CellYdimensionDeg = 0.00083333335351
ScaleinMeters = 1.0

[Source3]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "SRTM_19_03.BIL"
Lat = 50.00000000000000
Lon = -90.00000000000000
NumOfLines = 6000
NumOfCellsPerLine = 6000
CellXdimensionDeg = 0.00083333335351
CellYdimensionDeg = 0.00083333335351
ScaleinMeters = 1.0

[Source4]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "SRTM_19_04.BIL"
Lat = 45.00000000000000
Lon = -90.00000000000000
NumOfLines = 6000
NumOfCellsPerLine = 6000
CellXdimensionDeg = 0.00083333335351
CellYdimensionDeg = 0.00083333335351
ScaleinMeters = 1.0
;----------------------------------------------------------------------------------

With the INF file saved, you can now make a batch file to make the mesh:

resample 924160m.inf
tmfcompress 924160m.tmf 924160mComp.tmf
tmf2bgl 924160mComp.tmf 924160m.bgl
del *.tmf

Save this as 924160m.bat... then double-click on this BAT icon, and your mesh should start processing. An LOD5 mesh will process quickly.

;----------------------------------------------------------------------------------

Check the mesh with FSX' TMFViewer or LWMViewer ( by Jim Keir ). Zoom in and look at the corners! Does the mesh extend beyond the LOD5 bounds by a pixel? It should, and that indicates the area was covered fully. The dotted line shows the boundry, and your mesh should extend a 1/2 a pixel beyond this dotted line.

This shows there will be no gaps between adjacent mesh bgls. The mesh can go into a scenery folder. Remember to delete the 'file.dat' file in the scenery folder before testing in CFS2, or your mesh may not show. CFS2 will rebuild this file on loading the scenery.

;----------------------------------------------------------------------------------
;----------------------------------------------------------------------------------

LOD5-sized mesh at LOD 9 is good for relatively small areas, but for a world backdrop, CFS2 should have LOD2-sized mesh at LOD 7. These would be CST sized mesh files.

The exact same principles apply, with the same tools. Find the LOD2-sized bounds, expand them a degree each way, download the TIFs, process into BILs, make the INF and BAT files, naming the files as the CST4 regions ( as per LEC )... conventional for CFS2.

;----------------------------------------------------------------------------------
;----------------------------------------------------------------------------------

DEM data from N60 northwards and S60 southwards is missing from SRTM datasets.
You can use the older GTOPO30 data:

http://eros.usgs.gov/products/elevation/gtopo30/gtopo30.html

You need only download the northernmost and southernmost files. This data may not be better than the default CFS2 data, but it may be needed for padding to complete the sources for making mesh, as we ALWAYS need data sources larger than the area we wish to have as mesh. You'll need to convert these to BIL as well. This dataset can be loaded into Microdem by selecting the hdr file.

Let's say you want to make a huge CST file of the NW corner of the globe. In LEC, enter latitude of 89 and longitude of -179, LOD2 bounds, Generate:

North = 90
West = -180
South = 67.5
East = -150

Each northern gtopo file = 30* x 30* North to south we only need 1 DEM, as we can't get larger than 90* north, and 67.5 is well within the 608 southern bounds of the DEMs. But east to west, we need 2 tiles... -180 is as far as we can go, but moving east, we should add a DEM to avoid gaps in the final mesh.

We need W180N90.HDR, and W140N90.HDR. Each of these can be loaded into microdem, and converted to BIL, as above. If you've downloaded all the northern and southern sets of GTOPO30, you might as well convert all to BIL ( perhaps into a new folder for future use ). Try using Edit/DEM Holes/Missing data to sea level, before saving as BIL.

cst4000.inf:
;----------------------------------------------------------------------------------
[Destination]
; the lod could be 5, 6, 7, 8, or 9
LOD = 7 ;-------- don't go higher than lod7 for a CST file or your framerates will suffer.
DestDir = "."
DestBaseFileName = "cst4000m"
UseSourceDimensions = 0
; exact extents from LEC
NorthLat = 90
SouthLat = 67.5
WestLong = -180
EastLong = -150

; 2 16 bit binary files from microdem

[Source]
Type = MultiSource
NumberOfSources = 2

[Source1]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "W180N90.BIL"
Lat = 89.99583594062193
Lon = -179.99583333333334 ; --------------from the BLW file
NumOfLines = 6000 ; --------------from the HDR file
NumOfCellsPerLine = 4800 ; --------------from the HDR file
CellXdimensionDeg = 0.00833333376795 ; --------------from the BLW file
CellYdimensionDeg = 0.00833333376795 ; --------------from the BLW file
ScaleinMeters = 1.0

[Source2]
Type = ElevS16LSB
SourceDir = "."
SourceFile = "W140N90.BIL"
Lat = 45.00000000000000
Lon = -95.00000000000000
NumOfLines = 6000
NumOfCellsPerLine = 4800
CellXdimensionDeg = 00.00833333376795
CellYdimensionDeg = 0.00833333376795
ScaleinMeters = 1.0
;----------------------------------------------------------------------------------------------

cst4000.bat:

resample cst4000m.inf
tmfcompress cst4000m.tmf cst4000mComp.tmf
tmf2bgl cst4000mComp.tmf cst4000m.bgl
del *.tmf



Double-click the BAT file, check in TMFViewer or LWMViewer. Install to the sim. Done.

;----------------------------------------------------------------------------------------------

One last scenario involves using a combination of coarser GTOPO30 and more refined SRTM DEM data sources. This would be needed, for example, an LOD2-sized LOD 7 mesh covering 67.5*N to 45*N, as ther is no SRTM data north of 60*N.

You can use overlapping data of different cell dimensions in a multisource INF, and resample will sort it all out for you. In this way you can use the best available data, and still get full coverage for your BGL.

There are 96 CST regions ( LOD2-sized ). LOD7 mesh for these should take up about 500 Mb total storage. This is a good base mesh for CFS2. Areas of interest could have LOD5-sized mesh, along with LWM and VTP files to define land/water, beaches, roads... and would take up about 30 Mb per area ( 6144 LOD5-sized areas for the world... 184.320 Gb of file space ).

Covering the world with very flyable LOD2-sized mesh at LOD 7 is possible if we find online storage to handle 500 Mb. I can have the files ready in a couple of weeks.

Dick

 

[MaskRider]

Thank you Dick. This is much needed. It will take me a while to get it digested.

Chris

 

[rhumbaflappy]

Hi Chris.

The important things to insure good mesh coverage, is to make sure your data source bounds is larger than your destination bounds And... good data will mean good mesh. Using multisource simplifies the data handling, and doesn't usually require expensive GIS programs to combine the data.

LOD 9 or LOD 10 might look nice and bumpy, but it will usually only be good for screenshots. The framerates need to be saved for missions, AI, and gsl objects.

I'm finding some new sources for mesh from Canada, Alaska, and using:

http://www.viewfinderpanoramas.org/dem3.html

for enhanced SRTM data. This gives pretty good coverage above 60*N, and better handling of mountain ranges. Still looking for Antartica at better resolutions.

The above tutorial shows a good freeware approach to mesh development. But I still like GlobalMapper to handle my GIS needs.

Dick

 

[Jean Bomber]

Thanks Dick,

I will copy the post .....

JP

 

[xavierb]

Dick,
Great tutorial and, again, I bow to your skills and knowledge. I think that 500 MB should not be an issue here at SOH considering the high value of this add-on. Have you already discussed it with Jagdflieger and/or other SOH staff?
Also I suggest (although I guess you have already would have thought about it) to divide the mesh in small files and create somekind of world map or other useful reference index to be able to download only the specific parts of interest for the end-user.

 

[rhumbaflappy]

Some ground work needs to be done with CFS2 first.

dem4km.bgl needs to be updated... that's the base LOD3 mesh. This file needs to be updated, and the default has a problem in Antarctica. Done.

Then the LOD5 mesh needs to be built worldwide... CFS2 only had a limited set. These can be grabbed from FS2002, but not too hard to rebuild them from more recent data. Again, Antarctica has a problem. Done.

CFS2 doesn't have any LOD7. This needs to get built. Working on it for the next few days.

Then, CFS2 needs updates of default LOD9 sets used, and additional sets added as required. I'll update the default areas, and then see what happens. I'll develop a set of 96 BIL files using CGIAR data, "panoramas" data, SRTM30 data, and GLOBE data.

Panoramas:

http://www.viewfinderpanoramas.org/dem3.html


Dick

 

[xavierb]

Looking forward and closely to your project, Dick. I am in process of building other airfields (yes, after almost three years) and elevation is a critical parameter as you can suppose.

 

[rhumbaflappy]

The default solomon9.bgl is LOD 9, but the data has very little definition. It's about 3 MB in filesize, and covers a bit less than an LOD3-sized area.

An SRTM-based LOD 8 mesh, although blockier, contains much more data definition, over 4 times the area, at a cost of about 5 MB filesize.

The views from the sim tells the story better. Same framerates, much better definition due to the detailed data.
Using LOD 8 still makes room for ultra-detailed LOD 9 mesh if desired.

LOD7 might be a bit too blocky, though the detail is great, and it doesn't look too bad in the sim... still better than the default LOD 9. Filesize of the LOD 7 is about 1.5 MB and still is covering 4 times the area of the default.

I have found that CFS2 has a heavy mesh interpolation, so using LOD 7 and LOD 9 together results in some elevation popping, whereas LOD 9 alone has little popping.

LOD 9 mesh covering 4 times the area, also is 17 MB... 3 times the filesize. And the LOD9 is more likely to show source data defects.

All things being equal, it looks to me as if LOD 8 mesh using SRTM is the way to go, with area coverage of the 'cst' ( LOD2 ) area. There are 96 of these areas in the sim... but only about 18 of theatre significance between Europe and the Pacific.

Dick

 

[rhumbaflappy]

A further note:

LOD8 mesh for the CST-sized areas makes files much too large to be practical.

A world mesh set needs to be scaled back to LOD7 ( about 300 meters/pixel ). Areas of interest can then be detailed to LOD8 or LOD9 as desired.

Mesh above LOD8 might reduce the sim the a slideshow in some areas, and really isn't for a combat sim. I'm thinking if you want pretty screenshots, FS9 or FSX is a better medium.

Dick

 

[xavierb]

I agree, Dick. Just tell you I have converted two GeoTiffs and processed them to a bgl per your instructions and the outcome is a 49 MB. file (LOD9). Too large to be practical.

 

[rhumbaflappy]

Hi Xavier.

I've got an LOD7 that is 61 MB... and I'm not nearly done with them all. So some areas may need to be split from LOD 2 size to LOD 3 size.

I've found I can process 6 'CST' areas per day from my collected sources. Best case processing is going to be 16 days of computer work churning out the BGLs.

I've backed away from CGIAR data, as I'm using GlobalMapper, a commercial tool. What I'm doing is using the same sources CGIAR used, and a few more I've located. I'll include the source list when I'm done.

I don't think theere's anything wromg with the CGIAR data, but there is a pixel shift from the original SRTM data. That means one more layer of resampling added to the data before it gets to be a BGL. More resampling means the data is probably further from the truth.

I'm also using a new line in my destination code:

[Destination]
; the lod could be 5, 6, 7, 8, or 9
LOD = 9
DestDir = "."
DestBaseFileName = "924160m"
UseSourceDimensions = 0
; exact extents from LEC
NorthLat = 45
SouthLat = 42.1875
WestLong = -90
EastLong = -86.25
SkipZeroBlocks = True

That SkipZeroBlocks line leaves out data from the BGL that is just empty water... and can save a lot of file space.

Dick

 

[rhumbaflappy]

This is the default MS FSX NW Guadalcanal:

Here's my LOD9:

I don't know for sure, but it looks like they used 30 arc second GTOPO30 data and oversampled it to LOD 9. Even my LOD 7 mesh should look better.

I wonder if I need to do a set for FSX as well?

Dick

 

[xavierb]

Thanks for the 0 blocks trick. Will try with it.

 

[rhumbaflappy]

I hope to have all the world at LOD7 by the next weekend. Also a replacement for dem4km.bgl, and a full set of the default LOD5 mesh omitted by Microsoft ( named like dw000n90.bgl ). Both of these sets are updates based on SRTM data.

The mesh is based on the original SRTM v2 ( finished ) 3 arc second data where available. Other data is used to fill voids, and add data to areas not covered by SRTM, and to correct problems.

The sources are SRTM 3 arc second, SRTM30, NED 2 arc second Alaska data, Geobase.ca 250m ( Canada mesh ), and viewfinderpanoramas mesh. I'm also awaiting permission from viewfinderpanoramas to use that data.

This set doesn't use the CGIAR data, as the half-pixel data-shift I discovered indicates a further 'distance' from the original data. I may end up using this set, if the originals are too full of errors, and CGIAR's set has the errors corrected. This interpolation of 1/2 pixel is about 45 meters dislocation... the data should be in the right spot, but it would be interpreted data rather than the actual SRTM measurement. The SRTM 3 arc second itself is interpreted from the original 1 arc second data, and that data was altered to reflect real world elevations, and known water levels. Bla, bla, bla... Resample itself interpolates the data to the LOD7 grid, so all this may be a minor objection.

I also need to test the full set in the sim, and then I can ask for permission to upload this rather massive set to the outhouse ( about 1.1 GB compressed ). I'm sure this set will have errors, and I'd be willing to correct errors, and upload them, perhaps twice a year until the complaining stops. I consider errors as spikes, pits, and border seams. Flattening can be best done with SCASM Area16n flattens, and don't constitute a data error.

Dick

 

[Jean Bomber]

I hope to have all the world at LOD7 by the next weekend. ...............Bla, bla, bla.................
I also need to test the full set in the sim, and then I can ask for permission to upload this rather massive set to the outhouse ( about 1.1 GB compressed ). I'm sure this set will have errors, and I'd be willing to correct errors, and upload them, perhaps twice a year until the complaining stops. I consider errors as spikes, pits, and border seams. Flattening can be best done with SCASM Area16n flattens, and don't constitute a data error.

Dick

thanks Dick for all your work:friday:

JP

 

[rhumbaflappy]

I've gotten permission from Jonathan de Ferranti to use his corrected SRTM data for a freeware mesh set ( viewfinderpanoramas ).

Jonathan urged me to use an LOD9 mesh for mountainous areas. Looking at the way CFS2 interpolates mesh on the fly, this may actually look better in the sim. But skipping an LOD7 mesh, and going with LOD9 only, would increase the file set from about 1 GB to about 16 GB.

I would also have to go from about 92 files ( 4 CST areas have no data due to ocean ) to about 1300 files minimum, as the data would be much too large to download.

So, I think I'll stick with an LOD7 set, so everyone can have decent mesh at that level. And I can provide further mesh at LOD9 for mountainous areas as LOD5-sized uploads.

I have about 250 GB of elevational data, plus the CGiar set I haven't used... another 13 GB.

I definately can produce LOD9, but the website bandwidth would probably prevent this.

I do think an LOD9 only set would be superior to using an intermediate set of LOD7 plus LOD9... the sim seems to do a good job of interpolating the higher mesh without need for the in-between set. This is contrary to most thinking about mesh, and I don't know if this is something only with CFS2, or if FS2002 and FS2004 also only need LOD9, skipping the lower meshes at LOD5 and LOD7.

The intermediate sets seem to increase the mesh "popping"... at least to me.

Testing the LOD7 mesh now, and it looks pretty good, but as Jonathan noted, the LOD9 looks less "rounded", which was his reason for producing the corrected data to begin with.

There are no gaps between the LOD2 areas... all data overlaps the adjacent LOD2 by 1/2 pixel as they are supposed to. I'm sure there will be 'groves' and dropoffs, as this is characteristic of SRTM in spots. If I see bad areas, I will see if CGIAR data ges rid of it, or else some hand-tweaking may relieve it.

Justin Tyme ( FSGenesis ) already has done LOD9 for the world, and I think it may be available at his website for sale. For $40 US you can buy the world at LOD9-LOD11, and there is a world buffer set for $15 that is LOD7 and LOD8. You might want to contact Justin before buying, to see if his sets work in CFS2... that would require compilation with the FS2000 resample.

Basically, I'm proposing a 'world buffer set' at LOD7, for use with CFS2, that would be very combat/framerate friendly.

Dick

 

[gavinc]

Hi Dick,
I dropped it to offer to help (not that I know what you are talking about but I can try to learn), however it looks like you are well on your way to getting it completed.

I think you are making the right call by doing LOD 7 across the board.

Thanks for all your work

Gavin

 

[rhumbaflappy]

Here's what I am getting...

In Google Earth:

In CFS2:

This is LOD7 as a base mesh, plus LOD9 from the same sources, with Slartibartfast watermasking, and some Area16n flattens neare the coasts. This is all pretty much automated. No hand tweaking. Covers and LOD5-sized area, like the default FS9 and FSX.

The masking is made only from the mesh, and CFS2Autocoast produces the coastal flattens... not sure if the flattens need to be made yet.

I'm going to experiment with Slartibartfast some more, as there is an option to add rivers by adding a list of spring points.

I'm not a big fan of VTP beaches. Slartbartfast cannot make them, but perhaps Sander de Cocq's CFS2Autocoast can, using Ground2K version4.

Dick

 

[Jean Bomber]

Wow !

The results are fantastic ,it's really something that will completly change cfs2 ,the masking from the strm source is the best way for a accurate placement of the LC WC and coast,the step upper of the SDC's work based on fs9 lwm,this give also to use the mesh plenly .

"I'm going to experiment with Slartibartfast"
I will .....

thanks

JP

 

[Snuffy]

Nice ... that looks pretty damned close to real!!

Congrats.