[NOTIFICATION] AGS version branch split

[Crimson Wizard]

That became apparent long time ago that developing AGS further becomes difficult while keeping the "luggage" of old functionality, which was mostly kept for running old games and easier project upgrade.
So this is overdue, and we are planning to introduce a "break" in the versions, where the updates are split into two "branches". We currently call these branches "ags3" and "ags4", but these are informal names, and may be given proper names later.

"ags3" is going to be a fully backward compatible version, based on the latest AGS 3.5.0. From now on it will only receive bug fixes, perfomance and utility updates. There will be no new script functions or game features whatsoever.
There's at least one planned update currently, which number will probably be AGS 3.6.0, and its main feature will be change from "Allegro 4" to SDL2 library. AGS used "Allegro 4" for nearly two decades, but it's discontinued, and often not very well suited for the contemporary OSes. As this change comes with new potential problems, there may be lesser updates in the middle (like 3.5.1 etc) when necessary, until SDL2 update works well enough to be officially released.

"ags4" will be a breaking change version, also based on AGS 3.5.0, but having all the deprecated functions cut off. It may also have other things removed, such as certain plugin functions support which are not convenient to keep. This will be the main branch to develop AGS further. It will also be updated to SDL2 library.
It's not known at the moment how it will be developed, there are several priority tasks, but it mostly depends on how much time contributors can spare, so it's too early to talk about any new features.
We do not yet know what number will this version use, but probably we'll skip couple of them to reserve for "ags3" branch, so somewhere in the range of AGS 3.7.0 - 3.9.0.


Related short-term roadmap ticket on github: https://github.com/adventuregamestudio/ags/issues/1121

[Snarky]

ags4
We do not yet know what number will this version use, but probably we'll skip couple of them to reserve for "ags3" branch, so somewhere in the range of AGS 3.7.0 - 3.9.0.

Wouldn't it be best to just make it AGS 4.0.0?

Some arguments in favor:

• It's a significant enough change that we want to very clearly distinguish it from earlier versions:
  
  
  
  • The new version has breaking changes in the API. While you can update older projects to earlier 3.x versions (though in some cases you need to enable some script compatibility options) without it breaking, you won't always be able to do so with this new version. It's much easier to remember the limitation if it's a new main version number.
  • If there is a ScummVM port of ags3, it will be confusing if it doesn't work with AGS 3.8.0 games or whatever. Again, much easier to communicate that it supports AGS v2.x and v3.x but not v4.x.
• Keeping two separate branches that increment on the same release number level is both confusing and a risk for the future:
  
  
  
  • There can be cases where you have version 3.8.5 as the most up-to-date "ags4" release, but if somebody looks at the release dates the most recent version is actually 3.6.0, and they won't have any clue why.
  • It constrains what release numbers you can use in the maintenance branch. What if someone fixes some major bugs or makes major improvements (e.g. to performance or compatibility), but we've run out of version numbers below the earliest "ags4" version (e.g. "ags3" is on 3.6.15 while the first "ags4" version was 3.7.0)? It will have to be a third-level increment even though it's a pretty significant upgrade.
• We're already talking about it as "ags4," it's in github as ags4, and at some point people will start making games with ags4 builds of the engine. To then jump back so that the unofficial ags4 builds are followed by AGS 3.8 (or whatever) as an official release is a recipe for confusion.

I'm guessing the reason you resist incrementing the main version number is that this is not the kind of fundamental revision of the engine you've had in mind for AGS 4.0.
I have a simple solution for that: AGS 5.0.

[ChamberOfFear]

I concur with Snarky

[Cassiebsg]

I agree with Snarky. Just call it 4. 

[Danvzare]

Yeah, just call it 4.
Sure the interface isn't getting an overhaul or anything, but it sounds like it's going to be enough of a departure to warrant the name change.
Besides, the informal naming scheme already suggests that most people see it as being a 4 anyway.

Also, what effect will switching from Allegro 4 to SDL2 have?

[Joseph DiPerla]

I "Believe" SDL has

*better OS Portability support
*is used more widely therefore can encourage more contributors
*is used by SCUMMVM
*is an easier library to implement and work with than Allegro.
*has faster 2D Blitting
*integrates better with other libraries
*I also "believe" SDL has better 3D routines.

This is just some of what I understand of SDL and I could be wrong about it all.

[Crimson Wizard]

I forgot to mention that we are also completely rewriting audio code to use OpenAL implementation based on SDL2.

Also, what effect will switching from Allegro 4 to SDL2 have?

Do you mean effect on users or effect on development?

The primary reasons for changing libraries were:
* better modern OS support,
* better device support (e.g. Allegro 4 did not have any native touchscreen support, and SDL2 does),
* because Allegro 4 is long discontinued we are stuck with ancient utility libraries based on Allegro 4, or have to write our own.

We still are going to use parts of Allegro 4 embedded into the engine, because SDL2 did not have alternative for everything (Allegro is just a more wholesome library), that's easier approach at the moment.
Raw bitmaps and drawing functions are prime example, as SDL2 has only rudimentary support for that (it's more focused on texture-based gfx).

[Joseph DiPerla]

I saw in the GIT Discussions that you are also rewriting the script compiler in c++. Was this originally part of the editor in C#? What is the benefit of the rewrite? BTW, nice job on that rewrite. I had been following along and it seems like a lot of work went into it with everyone involved.

[Crimson Wizard]

I saw in the GIT Discussions that you are also rewriting the script compiler in c++. Was this originally part of the editor in C#? What is the benefit of the rewrite? BTW, nice job on that rewrite. I had been following along and it seems like a lot of work went into it with everyone involved.

fernewelten did the rewrite on his own initiative. Script compiler was originally written in C and is now C++. The main reason for rewrite was to have a cleaner code, because old compiler is very difficult to read and understand, let alone improve.

[Joseph DiPerla]

Gotcha. I think I was thinking of AGS.Native.

[Snarky]

This is going a bit off-topic, but following some of the github discussions, I've seen claims that the rewritesâ€"both of the compiler and the backend switch from Allegro to SDL, I believeâ€"have led to worse performance. Is that correct, and if so, has it been addressed?

AGS already does not have great performance, so I would be nervous about an upgrade that degraded it further.

[eri0o]

It may have slightly worse if you are using the Software driver but honestly no one should ever. I can manipulate 3k overlays at 1440p on the OpenGL driver at 60fps.

If the game ran fine performance wise on my browser experiment, than you can expect it to run infinitely better native on your computer since that have debug stuff going on so I can read errors on the console. And is single thread and it's my hacky codebase...

In the end I expect things to have small problems in the short term and to getter better later - there's a good amount of TO-DOs and not much people to write some of the code, like, a Metal renderer would be a nice to have for MacOS - mentioning this because MacOS has dropped OpenGL support - iOS still does have OpenGL ES2 available.

Overall there wasn't many people using Allegro4 in 2020 but there are more people using SDL2 today, so I expect this lowers the platform specific problems that have been happening and needing attention to allow to have more time to concentrate in other parts of the Engine. It's also one less barrier of entry for people that want to starting looking into AGS codebase now and make contributions to it since there is a lot more written on SDL2 too.

[Crimson Wizard]

This is going a bit off-topic, but following some of the github discussions, I've seen claims that the rewritesâ€"both of the compiler and the backend switch from Allegro to SDL, I believeâ€"have led to worse performance. Is that correct, and if so, has it been addressed?

AGS already does not have great performance, so I would be nervous about an upgrade that degraded it further.

Where was the claim that compiler rewrite led to worse perfomance? I haven't seen such comment. There will be a switch in the editor for the first time, that toggles between old and new compiler.
Someone might do a perfomance test on a big complex script.

There was a notion that Software renderer has worse perfomance for non-32bit games, and I have suggestions why, but I haven't seen it myself only heard 1 person talking about it. The new software renderer is working about same way as it worked in mobile ports since the beginning (Android, iOS), but maybe there's some mistake or unnecessary operation that affects the speed. I did not address that yet because just finished the work on the port branch itself and did not have time to investigate.

There were also a problem that sound plays too loud in some games, and in one game I tested sound did not play correctly at all. Other games seem to run alright.

The SDL port is still a WIP, it's located in the wip code branch in the repository, and requires testing most of all.

It may have slightly worse if you are using the Software driver but honestly no one should ever.

At the moment Software renderer is the only way to run properly 8-bit games with palette effects, as other renderers are not made to update their textures when palette change (this is doable but was not a priority).

[eri0o]

Ah, I don't have the reported worse performance on software renderer or at least I haven't noticed it :/

[fernewelten]

that the rewrites […] of the compiler […] have led to worse performance.

I'm currently not aware of any situation in which the rewritten compiler emits less efficient code than the old compiler.

[ToxicTuba]

the 4 shows the huge leap and is easier to understand for users and all in all makes more sense to be called this. I leaves enough space for updates of the 3.6 branch. I'm all for four

[jumpjack]

Is this the only thread about AGS4? It is very old.
The only other one mentioning AGS4 is Engine & Editor V4.0 Alpha - Proposal, and an issue on github.

I would like to follow the ongoing discussion, if any, about new features going to be implemented  in AGS4.

I add here a link to all AGS4-tagged issues on github, for reference:

Open
Closed

[Crimson Wizard]

Is this the only thread about AGS4? It is very old.
The only other one mentioning AGS4 is Engine & Editor V4.0 Alpha - Proposal, and an issue on github.

I would like to follow the ongoing discussion, if any, about new features going to be implemented  in AGS4.

I add here a link to all AGS4-tagged issues on github, for reference:

Open
Closed

Yes, as ags4 branch gets more new major features, we should probably open a proper forum thread, explaining priority goals and current state.

The mentioned github issue "AGS 4.0.0 Milestone proposal" is probably most up-to-date ticket with goals.

The changelog describing new features may always be found here: https://github.com/adventuregamestudio/ags/blob/ags4/Changes.txt

The downloads may be found on our build server, if you find the latest entry mentioning "ags4" branch:
https://cirrus-ci.com/github/adventuregamestudio/ags/ags4

It's worth mentioning that the idea of what "AGS 4" is have changed over time. Originally (number of years ago) I had thoughts about more major overwrite there, but as time went, it became clear that we don't have enough workforce to handle any kind of overhaul. So currently the idea is to implement major changes in smaller steps, and allow to break backwards compatibility whenever it gets in the way.

The principle about compatibility is to guarantee support only for importing the latest 3.* project format (3.6.0 currently), but don't guarantee running games compiled with latest 3.*. In other words, AGS 4 should be able to transfer the 3.6.0 project, but earlier projects will have to be upgraded to 3.6.0 first.