Overlay.Zorder

[jumpjack]

My game is behaving weirdly again. :-)
I define ZOrder of all my overlays only once at room creation, but I see various overlays changing draw order while character moves around them: sometimes overlay A is drawn over overlay B, other times B gets over A while character moves around; how do character movements influence overlays zorder?

This is the web version (continuously changing, so it could be not working at all when you run it...):
https://jumpjack.github.io/Space1999Adventure/AGSdebug/  (press CTRL+F to unlock the character)

Note:
the room is drawn by layers: layer 0 for floor, then further 3 layers for various objects; the flickering objects, for example windows, are on layer>0.

I seacrhed for "ZOrder" in whole project, but it only occurs in room script, at room creation.

[Crimson Wizard]

I define ZOrder of all my overlays only once at room creation, but I see various overlays changing draw order while character moves around them: sometimes overlay A is drawn over overlay B, other times B gets over A while character moves around; how do character movements influence overlays zorder?

Hmm, unless that's a scripting mistake of some kind, this might be a sorting issue in the engine. Character movement influences only how character itself appears compared to other elements, not how elements appear in relation to each other.

Do you have a script example demonstrating this behavior? Alternatively, is it possible to download a game.ags file somewhere, to let me test this under debugger?
I will try to set up my own test too.

EDIT:
Here's my test game that creates N room overlays randomly in the room, in "Before fade-in" event:
https://www.dropbox.com/s/by18sil7hozd99z/test--360overlayssort.zip?dl=0
I do not observe same effect there. But maybe my game setup has differences.

Are you sure you do not recreate overlays repeatedly, for example, or change their Graphic, etc?

[Khris]

The parts that disappear and reappear seem to be in the same spot as a bunch of background overlays; is it possible they have the same z-order value? As in, cannot be sorted reliably?

[Crimson Wizard]

The parts that disappear and reappear seem to be in the same spot as a bunch of background overlays; is it possible they have the same z-order value? As in, cannot be sorted reliably?

This may be the case; I recall that in some separate case AGS also compared object ID, which corresponds to their order of creation. Perhaps same could be applied to the drawing sort. The problem here though is that ID sequences are not shared through all the object types (for example, there may be roomobject with ID 1 and character with ID 1, and overlay with ID 1).

To improve this situation in the future we'd need to setup a certain rule for how the sorting is done among all the objects which have equal zorder/baseline.

The solution for now may be to give these tiles slightly different ZOrder.

Another solution that could be tried here, in my opinion: merging multiple static tiles, that are located in the same position, on a single overlay, instead of creating a separate overlay for each tiny element.

[jumpjack]

Alternatively, is it possible to download a game.ags file somewhere, to let me test this under debugger?

I don't  know if web version of AGS can be opened and viewed in editor, anyway it's in the same folder of main page:
https://jumpjack.github.io/Space1999Adventure/AGSdebug/my_ags_game.ags
https://jumpjack.github.io/Space1999Adventure/AGSdebug/004.ags

[jumpjack]

The parts that disappear and reappear seem to be in the same spot as a bunch of background overlays; is it possible they have the same z-order value? As in, cannot be sorted reliably?

Yes, I assigned same Zorder to all objects which in "real world" would lie down on same floor tile. But they are drawn bottom-up, so I don't get why the final result is not as I would expect.

Guess I have to rethink my isometric algorithm.

[Crimson Wizard]

Yes, I assigned same Zorder to all objects which in "real world" would lie down on same floor tile. But they are drawn bottom-up, so I don't get why the final result is not as I would expect.

What would "bottom-up" mean, logically? The overlay sorting is only defined by ZOrder parameter. The sprite sorting in AGS is done using a standard std::sort algorithm, where if two or more elements have equal factor, their relative order is undefined. Their order of creation does not mean anything in this regard.
EDIT: hmm, I thought this is mentioned in the manual, but it's not, so i have to add a clarification...

To make their order defined, we'd need to introduce another parameter, that would be used when zorder of two elements is equal. Historically AGS does this for GUI: if two GUI has same ZOrder, then it will also compare their IDs. But it does not do that in rooms (well, I mentioned that above).

If you want them to be reliably "bottom-up", then there are at least two natural solutions:
1) Have them have explicitly different zorder;
UPD: note that zorder may be negative too, so in case of a floor, you may make floor tiles have sequences of zorder < 0, to ensure they never draw over characters and other elements.
2) Paint the stacking tiles on a single overlay (as much as possible without breaking sorting among other overlays and characters).
UPD: you may even draw floor tiles on a room background, instead of creating overlays.

[Khris]

These overlays are not drawn bottom-up, they are created bottom-up. They exist in memory, and each frame they are sorted together with other entities, then drawn based on that sorting order.

If two overlays have the same z-order, their per-frame order is based on the cosmic rays hitting the earth and the energy flowing through ley lines in that precise quantum moment, which makes them flicker to the front and back.

Like CW has suggested, features which are supposed to appear directly on walls / tables should have a z-order offset by 1 vs. their base tile so they will consistently appear on top of them.
You basically need a stack for each tile, then add one to the z-order for each additional layer on that specific tile.

[jumpjack]

Another solution that could be tried here, in my opinion: merging multiple static tiles, that are located in the same position, on a single overlay, instead of creating a separate overlay for each tiny element.

This is an interesting idea: rather than  drawing whole layer at once, I should draw all layers of each tiles at once.

Now:




Then:

Draw/create order:



ZOrder assignment:

[jumpjack]

 

If two overlays have the same z-order, their per-frame order is based on the cosmic rays hitting the earth and the energy flowing through ley lines in that precise quantum moment, which makes them flicker to the front and back.

  Funny... but I think this is not the case: if you watch carefully you can see that flickering is not random, it always happens at same moment on same overlay when player is in same position. There is something in the engine code which makes player position influence overlays draw order. 

[Crimson Wizard]

If two overlays have the same z-order, their per-frame order is based on the cosmic rays hitting the earth and the energy flowing through ley lines in that precise quantum moment, which makes them flicker to the front and back.

  Funny... but I think this is not the case: if you watch carefully you can see that flickering is not random, it always happens at same moment on same overlay when player is in same position. There is something in the engine code which makes player position influence overlays draw order. 

It's not literally "random", as in Khris's joke. The actual result is determined by the number of game elements, their z-orders, and std::sort algorithm. As character moves around, its z-order changes (with y coordinate), which affects overall sorting. Elements with equal z-order will be treated depending on a coincidental factor. This result is actually deterministic in a given environment, but it won't help to know how it's received, and should not be relied upon, that's the point. (Hypothetically, it may even depend on C++ library implementation used when compiling the engine, so different versions of engine on different platforms may have slightly different result.)

[jumpjack]

I changed my code as suggested above, no more flickering now. 

https://github.com/jumpjack/Space1999Adventure/tree/main/AGSdebug2

This solution also saves some of memory, using just 100 overlays rather than 400.

Code: ags

  ///// Build room tile by tile
  int layerNumber = 0;          
  int overlayIndex = 0;
  int tileIdInt[4];
  for (int isoX = 0; isoX < mapwidthTiles; isoX++) {
    for (int isoY=0; isoY < mapheightTiles; isoY++) {
      isoToCartesian(isoX, isoY,  tilewidth * (mapwidthTiles/2));
      tileIdInt[0]  = mapTiled[isoX].elements[isoY];
      tileIdInt[1] = mapTiled1[isoX].elements[isoY];
      tileIdInt[2] = mapTiled2[isoX].elements[isoY];
      tileIdInt[3] = mapTiled3[isoX].elements[isoY];

      DynamicSprite* tileLayersContainer = DynamicSprite.Create(24, 200); // will hold all tiles stacked on same isox, isoy location
      DrawingSurface *tileLayersContainerSurf = tileLayersContainer.GetDrawingSurface();

      /////// Build the room by stacking on same isoX, isoY coodinates all the tiles from all the layers, 
      /////// and storing them into a room overlay; overlay is drawn only once all tiles have been drawn in it.
      ////
      //// # Create temporary sprite "tileLayersContainer"
      //// # Gets its drawingsurface into "tileLayersContainerSurf"
      //// # For each (isoX,  isoY) location:
      //// #    For each layer:
      //// #      Paste the layer tile into the drawing surface, at an Y level depending on layer number (Layer 0: y=0,  layer 0: y=7, ...)
      //// #    Create a RoomOverlay from the final sprite drawingsurface, and assign it a ZOrder as follows:
      //// #    Should be like this...
      //// #              0        
      //// #            1  1      
      //// #          2  2  2
      //// #        3  3  3  3  
      //// #      4  4  4  4  4 
      //// #    5  5  5  5  5  5  
      //// #    ....                  
      //// #    But player should be placeable between levels,  so multiply this ZOrder by half tile height:
      //// #                      
      //// #              0        
      //// #            6  6      
      //// #          12  12  12
      //// #        18  18  18  18  
      //// #      24  24  24  24  24
      //// #    30  30  30  30  30  30 
      //// #    ....                  
      
      for (int currentLayer = 0; currentLayer < 4; currentLayer++) {
        if (currentLayer == 0) { // Draw floor tile under all tiles,  empty or not.
          non_flipped = DynamicSprite.CreateFromDrawingSurface(floorTileSurf, 0, 0, floorTileSurf.Width,  floorTileSurf.Height);
          DrawingSurface *non_flippedSurf = non_flipped.GetDrawingSurface();  // Put sprite graphic into surface to draw it onto container surface
          tileLayersContainerSurf.DrawSurface(non_flippedSurf, true, 0, TILEDOWN + YFACT + 12); // Paste tile into container      
        }        
        if (tileIdInt[currentLayer] > FACTOR) { // Tile to be flipped  
          tileIdNorm = tileIdInt[currentLayer] - FACTOR - 1; 
          flipped = DynamicSprite.CreateFromExistingSprite(tileIdNorm,  true); // clone the sprite to flip it
          flipped.Flip(eFlipLeftToRight); 
          DrawingSurface *flippedSurf = flipped.GetDrawingSurface();  // Put sprite graphic into surface to draw it onto container surface
          tileLayersContainerSurf.DrawSurface(flippedSurf, true, 0, TILEDOWN - 7*currentLayer); // Paste tile into container      
  //debugPrint(String.Format("FLIPPED Overlay n. %d is located at ISO %d, %d, has normalized id %d (original id: %d, flipped slot: %d) and Z=%d", overlayIndex, isoX,  isoY,  tileIdNorm, tileIdInt[currentLayer],  flipped.Graphic, roomOverlay[overlayIndex].ZOrder), false);
        } else { // Don't flip tile
          if (tileIdInt[currentLayer] !=0) { // Regular tile
            tileIdNorm = tileIdInt[currentLayer];
            non_flipped = DynamicSprite.CreateFromExistingSprite(tileIdNorm,  true); // clone the sprite to flip it        
            DrawingSurface *non_flippedSurf = non_flipped.GetDrawingSurface();  // Put sprite graphic into surface to draw it onto container surface
            tileLayersContainerSurf.DrawSurface(non_flippedSurf, true, 0, TILEDOWN - 7*currentLayer); // Paste tile into container      
  //debugPrint(String.Format("normal  Overlay n. %d is located at ISO %d, %d, has default    id %d (                                ) and Z=%d", overlayIndex, isoX,  isoY,  tileIdNorm, roomOverlay[overlayIndex].ZOrder), false);          
          } else {  // Empty tile (=walkable area)
  //debugPrint(String.Format("normal  Overlay n. %d is located at ISO %d, %d and is empty", overlayIndex, isoX,  isoY), false);                    
            tileIdNorm = 0;
          }
        }
              
        if (currentLayer == 0) {    // Draw walkable areas only for base layer        
  //      1
  //    3  2
  //    5  4
  //      8

  // ymargin needed above tile for better handling adiacent tiles occluding player
        int x1 = screenx;                          int y1 = screeny + YFACT - YMARGIN/2;
        int x2 = screenx + tilewidth/2;            int y2 = (screeny + tileheight/2 + YFACT)  - YMARGIN/2;
        int x3 = screenx - tilewidth/2;            int y3 = (screeny + tileheight/2 + YFACT)  - YMARGIN/2;

  // ymargin needed below tiles to prevent apparent player walking over objects
        int x4 = x2;                                int y4 = y2 + YMARGIN; 
        int x5 = x3;                                int y5 = y3 + YMARGIN; 
        int x6 = screenx;                          int y6 = screeny + tileheight + YFACT + YMARGIN;
      

        if ((tileIdNorm != 0) && (tileIdNorm != 10) && (tileIdNorm != 23)  && (tileIdNorm != 45)  && (tileIdNorm != 46)) {
          // 0 = walkable:
          //    10 = doors (debug: open/closed)
          //    23 = glass floor (used between stairs)
          //    45, 46 = stairs      
          // !=0 = not walkable
    
          walk.DrawingColor = 0; // Drill non walkable areas
        
          ///// drill nonwalkable holes in walkable area:
          walk.DrawTriangle(x1, y1,  x2, y2,  x3, y3);
          walk.DrawRectangle(x3,  y3,    x4,  y4);
          walk.DrawTriangle(x4, y4,  x5, y5,  x6, y6);
          /////////        
          
        } else {
          //// Draw special regions (stairs and glass floor)
          if ((tileIdNorm == 45) || (tileIdNorm == 46)) { // stairs: not directly walkable,  but player.z must be increased
            myRegions.DrawingColor = REGION_STAIRS;
            myRegions.DrawTriangle(x1, y1,  x2, y2,  x3, y3);
            myRegions.DrawRectangle(x3,  y3,    x4,  y4);
            myRegions.DrawTriangle(x4, y4,  x5, y5,  x6, y6);
          }
  
          if (tileIdNorm == 23) { // glass floor (slightly above base level)
            myRegions.DrawingColor = REGION_GLASS;
            myRegions.DrawTriangle(x1, y1,  x2, y2,  x3, y3);
            myRegions.DrawRectangle(x3,  y3,    x4,  y4);
            myRegions.DrawTriangle(x4, y4,  x5, y5,  x6, y6);
          }
        }    
      }
        //Wait(5);
    }
    roomOverlay[overlayIndex] = Overlay.CreateRoomGraphical(screenx - tilewidth/2, screeny - 2*tileheight + YFACT - TILEDOWN,  tileLayersContainer.Graphic,  0,  true);
    roomOverlay[overlayIndex].ZOrder = (isoX + isoY + 1)*6 + YFACT ;
    overlayIndex++; 
    } // isoY  
  } // isoX
  walk.Release();    
  Debug(5, 0); // show path of player  

[jumpjack]

I copied my "defective" .ags file into a folder containing the files of web engine taken from latest release (3.6.0 RC6), but the flickering is still there:

https://jumpjack.github.io/Space1999Adventure/AGSdebug3_/

I didn't keep the source of the defective .ags so I cant't compile it with updated editor.

Should the problem have been fixed on the editor/compiler or in the engine?

I thought that this fix included also the overlays ordering bug (undefined order if Zorder is the same):

- Fixed GUI sorting for cases when ZOrder is equal (this could cause both regression in drawing order, and click detection mistakes);

[Crimson Wizard]

I thought that this fix included also the overlays ordering bug (undefined order if Zorder is the same):

- Fixed GUI sorting for cases when ZOrder is equal (this could cause both regression in drawing order, and click detection mistakes);

No, this is not related to overlays. It fixes a regression in 3.6.0 related to GUI, that also use IDs when ZOrder is the same. This is the historical AGS behavior which we have to keep for backwards compatibility.

Besides, in your case, these are room overlays, and none of the game objects positioned inside a room (RoomObjects, Characters etc) have a reliable sort with equal baseline either. I mentioned this earlier in this forum thread, and explained what the problem is.

Whether this has to be changed or not is arguable, but "fixing" this may only be done if the mentioned problem is resolved; and definitely not in 3.6.0, because we're done making significant changes in it.

Meanwhile added respective notes to the ZOrder articles in the manual:
https://adventuregamestudio.github.io/ags-manual/GUI.html#guizorder
https://adventuregamestudio.github.io/ags-manual/Overlay.html#overlayzorder

[Snarky]

Whether this has to be changed or not is arguable, but "fixing" this may only be done if the mentioned problem is resolved; and definitely not in 3.6.0, because we're done making significant changes in it.

Since the draw order really is undefined when two things have the same z-order value (by definition), any ordering will be inherently arbitrary. Though I suppose there is a good argument to be made that it would be better to keep it stable (sorting by ID?) to avoid flickering—especially for characters, since that's the most common case where the player can directly affect the z-order, thereby making it hard for a game maker to ensure that two of them don't have the same value.

[Khris]

I agree that in any instance, z-sorting should fall back to the ID, and a lower ID should be drawn first.

[Crimson Wizard]

Something that I mentioned above, and which is a main issue for comparing IDs: different types of game objects do not share same ID sequence. Characters, Room Objects, Walk-behinds, GUIs, Overlays: all of them have their own sequence of IDs. Sure, it's possible to make Characters with the same z-order compare IDs among themselves, but what if Character and RoomObject have equal z-order? Or RoomObject and Overlay? Or Overlay and GUI? Which one goes first then? Then we need to either hardcode the priority based on a object type, or find some other solution.

Besides that, Overlays in particular right now don't have same kind of explicit ID as other objects, their IDs are internal handles arranged in a specific way, they have "special values" among them (for overlays autogenerated by the engine), and overall afaik do not always correspond to the order of creation. Hence the result of their comparison will not be "expected" at all times either.

These are at least two problems that need to be resolved.

[Snarky]

I would say it does not matter what the rule for sorting between things with the same z-order is, since as already mentioned it is arbitrary. There is no "right" or "expected" way to sort them, and it's not even important that the sorting is predictable. The only "problem" we might want to solve is flickering (i.e. that the sorting is not stable). So any hard-coded ordering between different types, for example, is fine.

As for overlays, they are always explicitly created with a given z-order, right? In that case, I don't think it's even much of a priority to ensure stable sorting at all: just let the game maker be responsible for ensuring that two of them don't have the same z-order.

To put it succinctly: the way for a game maker to control the order of drawing between different things is the z-order. If two things are given the same z-order, it's up to the engine to decide, based on whatever it likes.