Cityscape

[diki]

Hi,
I've been working on creating a simple Cityscape with ZGE; it's supposed to be procedurally & randomly generated (of course) and contain basic collision detection (since it's going to be the level for the game). I've already made some small progress that I'd like to share here.

The first program generates a set of building blocks, each containing 9 randomized buildings, separated by sidewalk & street; I just tried to get a nice variety of looks from similar source materials. Apart from some blend-mode related glitches and the uniformity of the scene, I quite like the look.

The second is a simple implementation of a recursive area-splitter (this probably has a specific name to it, which I don't know); it starts out with a single square area and splits it into 2 uneven halfs everytime the Space key is pressed. The next time the space key is pressed, each of these is halfed again ... until there is a nice fractal (?) landscape. With some tweaking and conditioning, I've already gotten it to produce results that are very close to what I need.

The issue I am currently facing is performance, mostly; the building block-demo starts to impact performance at a size of 5*5 blocks ( 225 buildings, each with several nested conditions in the OnRender-component), the splitter starts acting up at around 512 areas (each one using 3DBox collision mode). My laptop is admittedly quite weak on the chest, but I guess these kinds of problems will appear everywhere at some scale. Having the entire City rendered & collided at once will probably be impossible at reasonable framerates, so I've been thinking about workarounds. I would be glad for some input here, since some of you are even familiar with ZGE's source and could put them in perspective more easily. They are:

• 1) using fully renderd & collided models only for the buildings immediately around the player, replacing all other buildings with dummy-models(that spawn the corresponding building when the player get's close)
  
  2) using billboarded sprites instead of 3d geometry for those dummy objects that are close enough to still be viewable (all other dummy objects would just be empty)
  
  3) separating collision & rendering: using actual models for the collision-detection of every building, but using only one single unified model for rendering all the geometry (by cycling through an array and rendering on demand). This one sounded better when I thought it up - the basic question is wether the amount of Model-objects actually impacts performance, and if any of that can be saved by some method.

It would be cool to hear your thoughts on this!

[Kjell]

Hi diki,

Nice progress so far ~

Let me try to address some of the questions you have. But first, be careful that you don't aim too high .. pulling off a open-world environment ( like in Grand Theft Auto ) with a decent framerate is no walk in the park. Anyway ..

There's no way to turn collisions on / off in real-time at the moment ( Francesco has requested this though ) .. so unless you write your own collision library ( you probably want to do this ), you could create 2 model versions of each Building, one with collision and a high polycount .. and one without ( low poly / billboard ).

There is no build in culling feature ( has been requested ), but since you're dealing with a city a cheap Axis Aligned 2D culling method will be sufficient ( as long as you won't be able to stand on top of buildings or fly around in a helicopter ).

And no, having 1000 Models isn't that much slower then having a Repeat component with a 1000 loops.

K

[jph_wacheski]

Wow, some very fine work sir.

I just took a quick looksee but here are my first impresions;

I did not know that using a collision box of 0,0,0 would work to collide at all,. and in the subdivision example it selects with super fine res. nicely. I went quite far,. ~9k and it was still reasonably quick,. . seem like a lot of collision checking happening,. but im not sure what you intent that for,. .

the city one is quite sweet,. I'm wondering, why not just have each building as an object with its collition setup? Should be reasonable for a city of this size, (based on the FPS example I did, that had quite a lot of box3d structures,.) yeah its the simplest possible thing to do,. obviously if there is no jumping flying 2d collision would work fine,. . the thing with swaping stuff in and out, based on position, is what about the non-player entities? I suppose a script for the player would work on them as well,. .

sorry, no real help here,. just some thoughts on what Im seeing. perhaps I will have a hack at the city later and see what more I think of,. . Thanks for sharing, interesting work indeed. I look forward to playing something!

[VilleK]

Yes, I agree this is very interesting. Nice work!

It is always difficult to determine which part of the game is being most performance-heavy. I usually try to remove stuff until I notice a great speedup. For instance removing the "DefineCollision" in your splitter-project does not increase speed much. Removing the rendering components on the splitter-model gives a x2 speedup. The remaining cpu-load is probably running the OnUpdate-components (condition with expression) on all models.

The building-block project has lots of logic in OnRender so it would definitely benefit from some culling mechanism. We have been discussing creating simple built-in culling in ZGE so maybe we can implement that now.

There are lots of textured boxes being drawn for every block, a faster approach would be to generate one Mesh for every block using MeshTransform to position individual buildings, and vertex colors instead of textures. This would be much faster because the mesh can be drawn in a single OpenGL call and you also avoid the state-changes from using several UseMaterial in a single OnRender.

Also I will run your projects with a profiler to see if I can find any obvious bottlenecks inside ZGE that can be optimized further.

[VilleK]

I ran the projects with the profiler. The idea of a profiler is to find where the application is spending most of its time. The results are approximate but they usually give a good indication.

Splitter-project with 4096 models:
25% running expressions
11% rendering code
2% collisions

Block-project with blockcount 10:
63% running expressions
8% rendering code (mostly using materials and rendering meshes)

In both cases it is clear that the ZGE-expressions are using the largest amount of cpu-time. This includes all expressions such as WhileExp on Repeat-component, Condition-component, ZExpression-component.

ZGE-expressions are not particularly slow but in these projects there are many models.

Splitter-project: 4096 models * 1 expression (Condition-test) * 60 fps = 245760 expressions per second.

Block-project: 100 models * ~50 expressions (lots of conditions in a repeat-loop) * 60 fps = 300000 expressions per second.

It is hard to say something general about this because if the meshes used were of a higher polycount it could be the rendering code that was the limiting factor. But in this specific case it would probably lower the frame rate to try to cull the meshes using more expressions than just drawing all of them! At least if you have a decent graphics card.

Since the block-models are static they would probably benefit greatly if ZGE used OpenGL display lists to render models, because then the OnRender-code with all the expressions would only be called once for every spawned model (once per application instead of once per frame). I can try to make a new "Static"-property on Model and see if this works.

[Kjell]

Hmm,

If you'd put all the OnRender Components of "BuildingBlock" inside a culling condition I'm pretty sure you'd get a significant performance boost ( it would just skip everything in OnFalse no? ).

But Ville is right when it comes to "parsing" things together that are static. Generate a single mesh for all of the buildings within a BuildingBlock, the same goes for Materials and Bitmaps. A single material and texture should be enough for all of your buildings ( we need the TexCoord property in MeshExpression! ).

K

[VilleK]

I tried to use display lists and noticed a speed-up, but there are several down-sides to using them:
- they are deprecated from OpenGL 3.0 (so support for them might dropped sometime in the future)
- they are not compatible (or lowers performance) in combination with VBOs
- they cannot save all OpenGL commands so there are problems with losing state changes in materials (buildings sometimes flashes in the block-project)

Because of these problems I don't think it's a good idea to use them so I'm not checking in these changes.

[diki]

Thanks for your interest & investigation into the topic, I really appreciate it! It's very helpful to know what keeps ZGE busy, and interesting to hear that it's mostly the expression count that is slowing things down... I'll continue with some test cases and investigate culling.

[VilleK]

Feel free to post updated zgeproj files later on if you encounter more performance troubles. With the profiler we can find bottlenecks and try to make it faster for the benefit of other users. I redid the tests today with non-debug version of zge-engine and got higher ratios for RenderMesh than before, but the point is still valid that zge-expressions take a significant amount of cpu-time.

[diki]

Hi,
I'm currently a bit busy, so I haven't had time to clean up the project files yet, but I'd like to post a quick status report (wall of text, to be honest) anyway:

I merged the city generator with the fps engine, so one can generate & see the results right away. The city generator now uses an array instead of models, which improved performance noticably. The rendering also uses that same array, basically looping through every element every frame, fetching scale & position and rendering a corresponding box. Thanks to Kjells culling example, this is running pretty smooth, even with large numbers of elements (probably already more than the game will require).

Some perfomance tests I conducted indicate the following (taken with a grain of salt, of course):
1) The difference in performance of single mesh vs multiple meshes (both invoking the same triangle count per frame, effectively making this a test wether multiple OpenGL-calls impact performance adversely) is not noticable.
2) The difference between rendering multiple meshes by running a loop in the OnRender component of the application vs spawning that many models each rendering a single mesh in their OnRender-component is also not noticable.
3) Empty models have a very minimal performance impact (0.0000005 seconds on average :P)

With these tests in mind, I chose to set the render engine up using an array with the building data & looping through this array - instead of spawning a model for every building. Although there is no immediate performance benefit, the array allows for more control: I plan to use two arrays for rendering, one containing all buildings, and one that only contains the subset currently surrounding the player. Each frame, only a small portion of the large array is checked for proximity to the player, which would then be copied to the smaller array. The smaller array is used for rendering & is looped and culled in it's entirety every frame. This way, the large array could hold a lot of data, without impacting the per-frame performance; at the same time, the view could still be rendered smoothly (rendering only a small number of buildings each frame). The delay this causes in the becoming-visible of building should not be noticable when using a large enough threshold distance.

More to come later, I guess :)

[diki]

Hi,
I'm busy with university at the moment, so I won't be able to continue the game for a while; and since it wouldn't have progressed that far without the help of the community, I'd like to post the current state - maybe other people find it helpful in some way.

The City-Generation part now has an interface to simplify generation; it will only be visible when hovering the mouse over the buttons. Pass Count is the number of 'splits' that are to be performed; you can set the size at which no further split should be performed on an area, as well as the random bias (a bias of 0 results in exact 50/50 splits). Lower & Upper Thresholds can be set to delete areas below or above a certain size; you can also specify after which pass/split they should be deleted for more flexibility. Street Passes specify up to which pass streets will be inserted in the gaps - three numbers for three street widths (broad to tight). Final Scale will scale the entire city (with streets retaining their size relative to the player).

Keys (Generator):
Buttons can be left-clicked to increase, right-clicked to decrease, or dragged to change parameters.
Space - same as 'Generate!'-Button
E - Reset
Q - Switch to Player

Keys (Player):
C - toggle third/first person Camera
WASD: Move
Right Mousebutton: Jump
Left Mousebutton: Fly (into view direction)
Q - Switch to Generator

[jph_wacheski]

Wow, very cool! I have not looked trought the code yet, but initial testing shows that it is working very well now. Perhaps I can learn some generation tricks and find some new ideas for use with my cavern generation tests,. .

Keep up your interesting work. What are you studying? (urban planning??)

peace.

[VilleK]

I like the easy-to-use user interface and enjoy flying around the city, momentarily switching to top-view to make the culling algorithm visible from above, very cool! Runs very smoothly here.

The source is very neatly organized and can will surely be useful for other ZGE-users. Thanks for sharing this impressive project!

[diki]

hi,
i don't want to repeat how busy i should be, so i'll just drop the current unfinished build which has working collisions and features saving/loading the generated city. the culling is currently not up to par, and the controls are wonky from a failed attempt at making them deltatime-compliant.

[Kjell]

Hi Diki,

Not sure how much detail you're going for, what kind of environment style and whether you're planning to generate everything dynamically ( wouldn't recommend it ), but here are two free building / city genetators you can use or draw inspiration from.

Building Generator written in MaxScript.
City Generator written in Python.

K