SCOtopo - Modern Open Rails Cloud Terrain Builder (USGS 3DEP)

[scottb613]

Hi Folks,

It's truly time to provdie an alternative to DEMEX and its 32-bit limitations. I've been working on a new tool called SCOtopo for a - - - while. It is a modern, automated terrain building application designed to generate elevation data directly for Open Rails and MSTS-compatible route layouts. It focuses on utilizing 1m LIDAR data sets - still active with USGS.

Instead of requiring you to manually hunt down, download, and convert digital elevation files, SCOtopo reads your existing route structure and dynamically streams live elevation data from the USGS cloud service - to overwrite or populate your route's elevation grids (*_y.raw files).

USGS 1m LIDAR Coverage Map:

https://apps.nationalmap.gov/lidar-explorer/#/

================================================== ================
CORE FEATURES
================================================== ================

* Intelligent Multi-Resolution Fallback: It primarily requests high-resolution 1-meter LIDAR DEM data. If a specific area lacks 1-meter coverage, it seamlessly falls back to 1/3 arc-second (~10-meter) data, ensuring zero gaps in your terrain mosaic.
* Smart 1m to 8m Downsampling: Open Rails requires terrain data to be spaced out in 8-meter points. Rather than just throwing away detail to shrink the massive 1-meter USGS data down to fit, SCOtopo analyzes the elevation patterns of each 8-meter block and calculates a mathematically smooth average, giving you highly detailed hill contours without any rough, jagged pixel edges.
* Neighbor-Fill Stitching: If the USGS database has minor missing data holes, the program utilizes an automated bilinear neighbor-averaging algorithm to patch void spaces cleanly.
* Edge and Corner Blending: It evaluates shared boundaries across adjacent map tiles. It averages heights along horizontal and vertical borders (including corners) so that your route has completely seamless tile transitions with no artificial cliffs or tears.
* Advanced Geographic Mapping: It natively handles the legacy MSTS/Open Rails Interrupted Goode Homolosine world projection system, converting raw latitude and longitude lines accurately into native game tile coordinates.
* Non-Destructive Inspection Mode: Features an `--inspect` option allowing you to scan your route, determine geographic bounding boxes, and see missing height stats without writing any modifications to disk.
* Will not process the old legacy 4km tiles.

================================================== ================
ADVANTAGES OVER DEMEX
================================================== ================

Legacy utilities like DEMEX served the community well for decades, but they are tied to an outdated era of manual data processing. SCOtopo offers several major upgrades:

* No Manual File Sourcing: DEMEX requires you to manually browse external websites, download individual GeoTIFF or .dem files, extract them, and index them. SCOtopo handles data hunting and streaming completely over HTTP natively.
* Only downloads the data needed, not massive GeoTiffs, cutting downloads to more than half the size of doing it via the old method.
* Modern OS Compatibility: DEMEX relies on older Win32/VB6 frameworks that struggle on modern 64-bit Windows environments. SCOtopo is built on modern .NET core architectures utilizing standard, updated GDAL engine wrappers.
* Dynamic Data Resolution: DEMEX is usually constrained to a single file resolution per build pass. SCOtopo actively blends multiple resolutions (1m and 1/3 arc-second) into a single coordinate grid on the fly.
* Automatic Patch Math Alignment: While DEMEX can sometimes introduce tile gaps or require manual layout tweaking, this tool directly calculates and updates the complex internal binary token scales (TokenTerrainSampleFloor / TokenTerrainSampleScale) inside the .t files automatically.

================================================== ================
KNOWN LIMITATIONS
================================================== ================

Please keep the following restrictions in mind when using the tool:

* USGS Territory Restriction Only: The program is hardcoded to query the United States Geological Survey (TNM Access API). As a result, it only supports terrain generation within the United States and cannot build international routes.
* Hardcoded Dataset Dependencies: It relies strictly on the "Digital Elevation Model (DEM) 1 meter" and the "National Elevation Dataset (NED) 1/3 arc-second" datasets. If neither dataset covers the targeted area, generation fails.
* Internet & API Performance Reliance: Because the tool streams live data directly via GDAL virtual file systems (/vsicurl/), it requires a stable internet connection. Large routes can face long processing times or timeouts if the USGS servers experience high latency.
* Fixed Tile Layout Grid: The mathematical engine is locked to a grid size of 256x256 int16 samples and an 8.0-meter post-spacing format. It will not adapt to alternative grid formats or different terrain sample scales automatically.

================================================== ================
================================================== ================

While still a work in progress - I've managed my first successful run - a major milestone. As far as I can tell - it created a perfect terrain mesh for the desired area - an area I am quite familiar with as I've been building/modifying a route in this area for many years.

Attached is my test - so you can see what 1m buys you - no track - so if interested - just open in TSRE.

Regards,
Scott

TopoTest.zip

[slipperman]

Hi Scott,
This sounds to be a very interesting and useful project and I wish you good luck with it

One question - will it only be suitable for North American routes or will Europe and the UK also be covered?

Cheers,
Ged

[scottb613]

Hi Scott,
This sounds to be a very interesting and useful project and I wish you good luck with it

One question - will it only be suitable for North American routes or will Europe and the UK also be covered?

Cheers,
Ged

Hi Ged,

Thanks much. Getting this far was a long time coming.

The only free source I’ve ever seen for Europe is our “shuttle mission” SRTM data - 30m source - which is also what is used for “HGT” files used in TSRE. Do you have free 1m source data? One of the key points of this program is to pull only the data needed from cloud storage via the USGS API - honestly - it doesn’t sound promising. If a source materialized I could look at adding it. I also plan to upload my code to GitHub at some point. It’s coded in C# with the GDAL libs.

I still need to build some type of simple GUI and handle Distant Mountains as well. It’s CLI only - at this point.

Regards,
Scott

[slipperman]

Hi Scott,
I only had a very short association with route building a number of years ago and for that I, obviously, used the HGT files.

I'm sure there will be members who are, or have been, European or UK route builders. They should be able to let you know if 1m source data is available for that region.

Cheers,
Ged

[eric]

Sounds awesome.... would be awesomer if we could get 4-5m spacing on the output grids... I thought Demex could get to a finer resolution, and TSRE seems to recognize whatever the T file and _raw files are written with.

[scottb613]

Sounds awesome.... would be awesomer if we could get 4-5m spacing on the output grids... I thought Demex could get to a finer resolution, and TSRE seems to recognize whatever the T file and _raw files are written with.

Hi Eric,

I’ve used 1/9 arc/sec 3m data with DEMEX but the files for GEOTIFF are so large that they really becomes impossible to use - they also had limited coverage and were discontinued in favor of the 1m LIDAR.

Are you talking about increasing the ORTS resolution beyond 8m? That’s our limiting factor - right - ORTS is hard coded to 8m - I’m reducing the res to accommodate ORTS.

Regards,
Scott

[eric]

If that's a hard limit, so be it. I know that we had higher resolution terrain in the MSTS-RE, especially with the 3m resolution available for urban areas.

If you did a 3m or 4m grid.... would ORTS barf or simply normalize it to the 8m grid? Reason I ask is that for terrain sculpting, the 8m grid isn't very forgiving.

[scottb613]

Hi Eric,

I could certainly be wrong - my understanding - I think from Wayne - the terrain mesh in ORTS has data points 8m a part. No matter how much better the resolution of your source DEM - it will be reduced to data points 8m a part when running ORTS. Doesn't matter which tool you use. If you have less than 8m - say 30m - in ORTS the points will still be 8m but since your source is 30m - 3 points will have the same elevation data. I'm not sure how DEMEX or TSRE would handle the .75 points left over.

If I am wrong and someone knows better - please let me know - my code is jumping through all kinds of hoops to smartly downgrade 1m resolution to 8m resolution.

If the ORTS development team wishes to change the ORTS mesh to 1m - I can certainly accommodate them.

Regards,
Scott

[pwillard]

I applaud your efforts so far. This is great stuff.

[amtrak115]

So I'm going to ask maybe a dumb question....So I could build a route of an actual location but build it as a tabletop route....i.e. no elevation ...then run your code I would get 1m terrain elevated data...right??

Barry

[scottb613]

I applaud your efforts so far. This is great stuff.

Hi Pete,

Thanks - Just got tired of struggling with DEMEX choking on large DEM.

So I'm going to ask maybe a dumb question....So I could build a route of an actual location but build it as a tabletop route....i.e. no elevation ...then run your code I would get 1m terrain elevated data...right??

Barry

Hi Barry,

Yep - in my 20-mile test route, I went into TSRE, created a new route, created the flat terrain tiles from markers, saved, and exited TSRE. I then ran my SCOtopo program pointed at the route. It finds the existing terrain tiles (.t and *_y.raw), calculates their real-world position and coverage using the Open Rails/MSTS tile projection, then connects to USGS cloud elevation services.

It streams the needed 1m LIDAR elevation data directly from USGS cloud-hosted DEMs, without requiring the user to manually locate and download GeoTIFF files. Where 1m coverage is missing, it can fall back to the USGS/NED 1/3 arc-second data, roughly 10m resolution.

Before writing anything back, it builds the full route terrain grid in memory, fills small missing-data holes, and smooths shared tile edges so adjacent tiles match. Then it writes the elevation data back into the route’s existing terrain tiles. Launch TSRE again, and the terrain is fully realized.

It can append, meaning it only processes tiles that do not already have elevation data, or overwrite, meaning it rebuilds every terrain tile in the route. My 20-mile / 64-tile test route took roughly 4-5 minutes on a 350 Mbps internet connection.

Short term planned additions:
- Build terrain tiles from markers without first creating flat tiles in TSRE.
- For large routes on older machines - I need to batch the memory job, so I am not holding an entire route in memory while processing.
- Add a simple GUI for users who fear the command line.
- Generate Distant Mountain tiles in the same operation.

I do not plan to recreate DEMEX’s graphical route/terrain preview screen.

Thanks for the interest.

Regards,
Scott

[pwillard]

Personally, I have struggled with all DEM-related tasks (hence why I have produced 0 routes). It seems that there is often so much time between sessions for me that the solution I was using either disappears or changes fundamentally. Hopefully, your tools can keep up with the whims of the network-based sources.

[scottb613]

Personally, I have struggled with all DEM-related tasks (hence why I have produced 0 routes). It seems that there is often so much time between sessions for me that the solution I was using either disappears or changes fundamentally. Hopefully, your tools can keep up with the whims of the network-based sources.

Hi Pete,

Hah - yeah - I've always pushed for better tools for route builders - the easier we make it - the more routes will be made - if you build it - they will come. The effort to build a route from scratch - even a short one - is a herculean task undertaken by very few. I know ORTS only has a few developers - over the years - both Eric and I have tried to facilitate some teamwork between TSRE/ORTS - as the most impactful changes to TSRE would require corresponding changes to ORTS - but to the best om my knowledge - requests haven't been successful. I recall an old conversation with Goku expressing the same frustration on the lack of coordination.

This task was something I could identify and address - that would hopefully substantially improve ORTS (at least for US users) - without having to rely on anyone else.

Regards,
Scott

[scottb613]

Hi Folks,

Distant Mountains (DM) - we have two formats for Distant Mountains that are not compatible with each other.

DEMEX DM tile:
16km x 16km coverage
64 x 64 height samples
4,096 elevation points
sample spacing about 260m

TSRE DM tile:
32km x 32km coverage
256 x 256 height samples
65,536 elevation points
sample spacing about 128m

Did you know - TSRE DM tiles have a far greater density of data points than DEMEX - therefore provides a better more detailed terrain than DEMEX.

I did not know this until a day or so ago. Clearly, you don't want to use Distant Mountains created with DEMEX anymore. TSRE can create them via HGT files. My app uses the TSRE format and just makes it easier.

I have my preliminary GUI built - it processes Distant Mountains and I just ran it on a full route worth of tiles. You can select tiles via existing tiles, marker files, or the track database. When using a marker file or track database the Radius comes into play based on the respective anchor - 0 is only the tile the anchor is found on - 1 is a 3x3 tile matrix surrounding the anchor - 2 is a 5x5 tile matrix.


Capture.jpg


Regards,
Scott

[amtrak115]

That sounds like a plan and welcomed addition....I subscribe to the topic...hopefully that will let me know when you release...I have a few routes I want to try it out on.