DEMEX Replacement

[scottb613]

Orignally, DEMEX read the geotiff files as downloaded. Around 7 years ago, USGS changed the format and now if you try to import a raw geotiff, you get some error which I thought would be fatal until just on a whim, I read it into VTbuilder, and then exported it as a geotiff. When you do this the new file is slightly smaller than the original and so I ass/u/me that something is removed. The problem is that at any moment, and best as I could tell, no notification, USGS changes the format. So what then. Maybe VTBuilder would fix it, maybe not. Also, VTbuilder will not read the 1/3 arc-sec files, only the 1 arc-sec, so you cannot get really good resolution. In fact you might not get any better than .hgt files, but so far I have used only the geotiff files.

definition: Every Southern Rwy officer, no matter how low on the totem pole, was required to keep a little sign on his desk. ASS/U/ME This was to remind you that if you assumed anything you were making an ass out of yourself and the other individual involved. I kept it on my desk at CSX. There was another: YCSFSOYA Every officer was to represent the company as best as they could especially around shippers, so YCSFSOYA = You can't sell freight sitting on your ass.

Jerry Sullivan

Hi Jerry,

That's a new piece to the puzzle I didn't know - and that blows. I've used VTBuilder before.

Anything processed with VTBuilder converts 10m (1/3 arcsec) data into 30m (1 arcsec) data - - - not cool. Yep - HGT files are 30m.

I wonder if QGIS can maintain the resolution?

Hah - ASSUME was well known in the Nav as well.
😁


Regards,
Scott

[pwillard]

FYI: The file "ConvertXY.bas" in the RR source code correlates with some of the discussions in the 2001 thread that was referenced.

[scottb613]

Hi Folks,

Just ran a test.

Downloaded file USGS source GeoTIFF 456MB (No Load DEMEX) - loaded and saved with QGIS is 446MB (Loads DEMEX) throws a tag error but seems to load fine.

Processed the same file with VTBuilder and saved 446MB (Loads DEMEX) no tag error.

Are they both losing resolution?

Also - Goku appears to have built in some functionality for GeoTIFF into TSRE:
https://www.trainsim.com/forums/foru...-files-in-tsre

Regards,
Scott

[scottb613]

FYI: The file "ConvertXY.bas" in the RR source code correlates with some of the discussions in the 2001 thread that was referenced.

Thanks Pete!

Hah - we'll sleuth through this. I didn't know RR had anything to do with the terrain mesh.


Regards,
Scott

[wacampbell]

One additional consideration is the mapping projection. The geotiff will likely not use the same mapping projection as your route. This will cause a mismatch between the terrain features and the route compass's lat lon coordinates. I can't remember what projection MSTS/OR uses, but I think it is something wierd. Any DEM importer will need to give some thought to mapping projection.

Wayne

[scottb613]

One additional consideration is the mapping projection. The geotiff will likely not use the same mapping projection as your route. This will cause a mismatch between the terrain features and the route compass's lat lon coordinates. I can't remember what projection MSTS/OR uses, but I think it is something wierd. Any DEM importer will need to give some thought to mapping projection.

Wayne

Hi Wayne,

Thanks a bunch - I realize we're probably digging through cobwebs - but I'm sure you know more than all of us combined.



Regards,
Scott

[CSRX]

I honestly have no idea about most of these files and how they work, but I am really glad that my post has sparked a conversation, with possible solutions for a newer program. If we could come up with something that is open sourced like Open Rails, there is a good possibility we could have several folks be able to work together and accomplish this.

I never did use VTBuilder, Sean had mentioned for me to try it, and I could never get it to work, however I have used QGIS 3.16.11 and simply opened up the Geotiff in that and re-saved it with "New" at the end of the file name, and that has allowed Demex to open it up. It takes a bit of time, but it works. I just wish Demex could read the MSTS/ORTS install correctly on Windows 10/11 so I would have to search for the route manually and then create the terrain manually by saving one tile at a time. It's not bad on a small section, but once it gets more than 10 miles of track, it becomes a very overwhelming and daunting task.

[eric]

Also - Goku appears to have built in some functionality for GeoTIFF into TSRE

Yeah, not really. There's a placeholder, but all the code looks to be a copy of the HGT code. Nothing was enabled or commented out beyond that.

[scottb613]

I honestly have no idea about most of these files and how they work, but I am really glad that my post has sparked a conversation, with possible solutions for a newer program. If we could come up with something that is open sourced like Open Rails, there is a good possibility we could have several folks be able to work together and accomplish this.

I never did use VTBuilder, Sean had mentioned for me to try it, and I could never get it to work, however I have used QGIS 3.16.11 and simply opened up the Geotiff in that and re-saved it with "New" at the end of the file name, and that has allowed Demex to open it up. It takes a bit of time, but it works. I just wish Demex could read the MSTS/ORTS install correctly on Windows 10/11 so I would have to search for the route manually and then create the terrain manually by saving one tile at a time. It's not bad on a small section, but once it gets more than 10 miles of track, it becomes a very overwhelming and daunting task.

Hi Shawn,

Yeah - we could use some improvement in this area.

Yeah, not really. There's a placeholder, but all the code looks to be a copy of the HGT code. Nothing was enabled or commented out beyond that.

Hi Eric,

Thanks for looking.


Hi Folks,

So a whole tile is 2000m square.


Regards,
Scott

[pwillard]

TIP: There are also GDAL libraries for PYTHON in case C languages are not your "thing".

And also GEOTIFF stuff..

https://kipling.medium.com/using-a-g...s-1c1b0349551b

[ebnertra000]

...

Hi Folks,

So a whole tile is 2000m square.

Tiles are actually 2048m squares, with 0,0 being in the middle of the tile

[eric]

MOST tiles are 2048. The MSTS tile creator utility had the ability to generate 4096 tiles as well, which were used by some early creators, but with a loss of terrain resolution.

I'd have to look, but Distant Terrain tiles may have been 4096, 8192 or larger.... lower resolution isn't critical at a distance.

[scottb613]

Huge agreement from me.

Not sure if this code has any insights...

And here

Hi Pete,

Thanks - good to know.

Given the vast amount of data this processes we might do better with compiled code. In the past - I've written ksh/awk scripts to act as parsers for inputting reports into databases - reached a point where the scripts couldn't handle the load - called in a real programmer who wrote a C program that easily handled ten times the volume without breaking a sweat.

Compiled code also means any and all libraries will be included in the executable - negating any need to have the end user download and install any external applications - I think.

Regards,
Scott

[scottb613]

Tiles are actually 2048m squares, with 0,0 being in the middle of the tile

Hi Travis,

Thanks - at a minimum we'll be compiling all the information in one place. I think I read that the GeoTIFF's are referenced from the top left corner - any solution will need to reference both.

Regards,
Scott

[scottb613]

MOST tiles are 2048. The MSTS tile creator utility had the ability to generate 4096 tiles as well, which were used by some early creators, but with a loss of terrain resolution.

I'd have to look, but Distant Terrain tiles may have been 4096, 8192 or larger.... lower resolution isn't critical at a distance.

Hi Eric,

Sounds right - yeah - the DM Tiles are huge.

OBTW: Last night's upgrade seemed to remove the "Reply" button from posts - leaving only the "Quote" button. To the best of my knowledge - there is no longer a way to "Multi-Quote" posts for a response now.

Regards,
Scott