Direct comparison - head to head

[sniper297]

I hadn't really tried comparing Open Rails to MSTS before because it's not really ready - signals and AI traffic need to be activated to actually find out if any given activity works better or not. Here's one place where I CAN do a direct comparison, Bzak yard on PO&N 10. No signals or AI traffic involved, just a moderate sized double ended yard. As many of you know I always thought huge empty yards didn't look realistic, keyword railroad TRAIN simulator, not railroad track simulator. So I fudged around MSTS limitations with "static trains":

https://www.trainsim.com/vbts/showthread.php?239724-Static-trains

Seventh pic in that yard is Bzak yard when it was still unfinished, fake trains on fake tracks to get around MSTS bugs.

So let's do some destructive testing ( FIRE IN THE HOLE! ), open Bzak0800 in the activity editor, add real loose consists to the six real yard tracks, see what explodes.

In Open Rails 360 that looks like this;



Hmm, not too good, only 12FPS. So let's see how MSTS handles that;

2.jpg

I think we have a clear winner.

[longiron]

Jim,
Your HW specs would be helpful in the discussion, as well as the Debug HUD showing number of primitives. I'm sure different systems will have widely varing abilities to handle scale you're driving towards. (Pun intended)

[sniper297]

Eeek!

Oh sir, I couldn't possibly show you my primitives, what would people think?

Model: Dell DXP051
CPU: Intel(R) Pentium(R) D CPU 2.80GHz (2 CPUs)
Memory: 2046MB RAM
Operating System: Windows XP Professional (5.1, Build 2600) Service Pack 3 (2600.xpsp.080413-2111)
DirectX 9.0c (4.09.0000.0904)

Video: NVIDIA GeForce 8500 GT 512.0 MB
NVIDIA Driver Version: 6.14.0012.5721

Sound: SigmaTel Audio
sthda.sys Driver Version: 5.10.4823.0000

Hard Drive: Maxtor 6L160M0 147.8 GB

* McAfee Anti-virus
* McAfee Personal Firewall

Another test, this one I lamed down a bit so MSTS could actually run it without choking. Shadows off in both ORTS and MSTS, distant mountains in MSTS disabled cuz ORTS don't have distant mountains, everything else in both sims cranked to dat's all dey is and dey ain't no mo'. Full screen 1280x1024x32 for both.





Near as I can tell that's a fair comparison, altho there seems to be a bit more distance fogging in ORTS all the objects and trainsets are displayed. Anyone wants to try this, for the default Marias Pass download and extract the zip, copy to \Train Simulator\Routes, run the WhitefishORTS activity.

wftest2.zip

[OTTODAD]

Not so fast, pal !

This in MSTS still beats whatever, especially the distant mountains I have helped you with !

O t t o

[sniper297]

Izzatso?

You missed the point, with ORTS not having signals yet any comparison in an area with a lot of signals would be unfair to MSTS since MSTS is handicapped by having to display the signals. Same thing with distant mountains, ORTS don't have distant mountains yet, so adding the extra weight of distant mountains for MSTS to carry makes it an uneven comparison. Idea here is to see what the new kid can do when carrying the exact same load.





ORTS wins here, if I enable distant mountains in MSTS the difference in FPS would be even more since MSTS would have to process more data.

[stresstool]

Question for an OR team member: His FPS is showing 15 but the frame time is 15,691 milliseconds. How can that be? Seems to me it can't be both. Did you find a hole in the space-time continuum?

[twpol]

Question for an OR team member: His FPS is showing 15 but the frame time is 15,691 milliseconds. How can that be? Seems to me it can't be both. Did you find a hole in the space-time continuum?

Does seem a touch weird, though given the also high frame jitter I'd guess that the data (for all three values) either hasn't settled or is generally being unstable. If there are loads of sound sources, like wagons, the current release can get pretty jittery and it could just be that.

[OTTODAD]

ORTS wins here, if I enable distant mountains in MSTS the difference in FPS would be even more since MSTS would have to process more data.

Perhaps with your computer.

Did you see the 34 FpS in my MSTS screenshot with everything set as MAX and Distant Mountains on ?

With distant montains not showing in ORTS I get these FpS below.

O t t o

[sniper297]

What is the sound of one SigmaTel clapping?

Odd thing but in computer games the sound card can sound just fine, but affect the video display in strange ways. Back in the Days of DOS and Dinosaurs I had a 486 with onboard video (state of the art at the time) and plugged my old original 8 bit Soundblaster in. Gunship 2000 had an odd glitch where the helicopter "weathervaned", rotating it into the wind whenever realistic wind was turned on. Microprose tech support had no clue, but when I replaced the old sound card with a 16 bit Soundblaster II, it eliminated the problem.

Consequently I suspect many of my video problems with this system could be cured by replacing that cheap onboard SigmaTel sound chip with an actual sound card. Note the numbers in the other post showing Bzak yard in explore route mode, all those cars are fake "static trains" and therefore have no SMS files to load and process.

[sniper297]

Missed your post, Otto.

"With distant montains not showing in ORTS I get these FpS below"

Okay, now turn off distant mountains in MSTS and take the same screen shot, see what the FPS difference is.

[OTTODAD]

Okay, now turn off distant mountains in MSTS and take the same screen shot, see what the FPS difference is.

2 FpS.

[sniper297]

Here's the kind of comparison I'm talking about that don't actually tell you anything, Chicago Elevated V4 at 35th and State;





ORTS has an unfair advantage there, because it doesn't have carspawners yet, therefore don't have to process all that framerate hogging road traffic. Once the carspawners in ORTS are activated, THEN we do that comparison so we're actually testing under the same battlefield conditions.

Woops, Otto, we're posting at the same time again. Well, that's not a huge difference, but MSTS gains 2FPS from not needing to process the distant mountains. At 53FPS for the same scene ORTS still wins.

[stresstool]

Once the carspawners in ORTS are activated, THEN we do that comparison so we're actually testing under the same battlefield conditions.

To jog your memory, if you add -nocars to the command line running train.exe, you can make MSTS not show autos. Try it just to see what difference it makes to your MSTS FPS in that spot.

[sniper297]

Sorry, my memory is too arthritic to jog these days, gotta push it in a wheelchair. Anyway, that works, got rid of all them infernal internal combustion contraptions;

0.jpg

Not a huge increase, but 11 to 15 is a pretty good percentage. Pound for pound ORTS still wins that one. Be interesting to see how well it does when all the animation is turned on.

[muskokaandtahoe]

Just to add more grit into the gears there's one other factor you havn't taken into account and that's the bottlenecks on each PC. For instance Jim, in one of your screen shots, I noted the Render process was waiting 36% of the time. I dunno offhand if that means your CPU was being overworked (I do suspectthat's the case) or your video card was but clearly a third of the time the software was sitting on it's butt waiting for some bottleneck to clear. Agreed that most every PC has a bottleneck somewhere and that the magnitude could vary by route and within route by location... but for purposes of examining what both programs can do you're still at the starting gate.

The ideal would be a controlled experiment where you swapped out just one element at at a time and remeasure. Say, fer instance, your 36% wait time was due to the video card being choked by too much work. Slipping in a better card might raise Open rails by 33% but 2% in MSTS 'cus MSTS doesn't ask the video card to do much of anything. Slipping in a great video card might not change those numbers because now the bottle neck has moved -- from video to CPU and that spiffy new video card is not waiting for the slow CPU to do something. Flip the changes around and drop in a better CPU and you might just see both programs performance jump by similar amounts (as MSTS is almost always CPU bound). Or not if hte new CPU is multi-core and the old one isn't.

My point is that absent such controlled experiments these head to head comparisons are only good for the person who takes them and perhaps the one other individual in 10,000 who has an identical rig. Every PC has it's bottlenecks and how those will affect two different programs is going to be unique to that PC.

As for upgrades, I recently replaced my AMD 4800+ / nVidia 7900GTX PC with an Intel i7930 / ATI 5850 and observed in almost every case Open Rail performance improved by a factor of 3. Adding a Win 7 gadget that monitors CPU and another that monitors the video card I noticed in one heavy duty scene (>7,000 primitives) the video card was at 95% utilization... something I noted w/ interest. My point in recounting this is that even w/ a brand new, rather well performing PC, a bottleneck of one sort or another is always lurking in the shadows and when it jumps out whatever effect it has will be unique to the PC