Playing audio on specific frames from scripts (not from the view editor)

[Gal Shemesh]

Hi everyone,

I'm working on a game with digitized actors (a lot of frames) and willing to play certain audio clips such as gun shots and other effects on specific frames.

I'm aware that I can use the view editor for adding sound at the exact frames that I want, but the process becomes very exhausting when dealing with loops that have dozen or hundreds of frames in them, as seeking for each frame in the view editor is way too slow, and also requires me to leave the script I'm working on and to go back to editing the view and slowly look for the frame I set a sound on for tweaking or changing it.

Therefore, I'm willing to call for the audio files to play from within my scripts, where I could easily tweak them as I want in a much faster way.

I've set my view animation cutscene to run at the delay of 3, and then under the room_repExec() function I put a check for the frames that I wish to play audio files on, like this:

Code: ags

function room_RepExec()
{
  if (oEgyptRun1.Frame == 126)
  {
    aGUNSHOT1.Play(eAudioPriorityNormal, eOnce); // first gun shot
  }
  else if (oEgyptRun1.Frame == 149)
  {
    aGUNSHOT2.Play(eAudioPriorityNormal, eOnce); // second gun shot
  }
}

The problem is, the audio clips play multiple times instead of only once, which I trust is due to as long as the frame I'm checking is showing on the screen (based on the delay that it runs in).

Is there a way to make it play the audio clip only once?

Thanks

[Nahuel]

I would recommend having a boolean to run it one-time-only.

Code: ags

bool soundedOnce;
function room_RepExec()
{
  // Reset the state after 1 Frame of the sound
  if ( oEgyptRun1.Frame == 127 || oEgyptRun1.Frame == 150 ) soundedOnce = false;
  if (oEgyptRun1.Frame == 126 && !soundedOnce)
  {
    aGUNSHOT1.Play(eAudioPriorityNormal, eOnce); // first gun shot
    soundedOnce = true;
  }
  else if (oEgyptRun1.Frame == 149 && !soundedOnce)
  {
    aGUNSHOT2.Play(eAudioPriorityNormal, eOnce); // second gun shot
    soundedOnce = true;
  }
}

With that the only time will sound is with soundedOnce = false;

[Gal Shemesh]

Thanks @Nahuel! That will work. Though this route will require me to make this bool for all frames I want to play audio in when calling to reset the bool, making the code too cumbersome.

Any idea of a way to check whether a specific sound file is actually playing, and to combine this with the frame check to make it play only once, just like checking if the bool is false?

[Nahuel]

Thanks @Nahuel! That will work. Though this route will require me to make this bool for all frames I want to play audio in when calling to reset the bool, making the code too cumbersome.

Any idea of a way to check whether a specific sound file is actually playing, and to combine this with the frame check to make it play only once, just like checking if the bool is false?

No worries Gal yes you can, but you need to change the approach, you need an AudioChannel

Code: ags

AudioChannel *audio_channel;

function whatever()
{
  audio_channel = aClip.Play();

  if ( audio_channel != null && aClip.IsPlaying )
  {
    // Do stuff
  }
}

[Khris]

Edit: nevermind, you can use Game.GetViewFrame() to grab the view's frame, then use .LinkedAudio to set the clip.


You can use a struct to assign the sounds:

Header:

Code: ags

struct ObjectFrameSound {
  int room, oID, frame;
  AudioClip* clip;
  AudioChannel* channel;
};

import void AddFrameSound(this Object*, int frame, AudioClip* clip);

Add this to the Global Script:

Code: ags

ObjectFrameSound ofs[20];
int ofs_count;

void AddFrameSound(this Object*, int frame, AudioClip* clip) {
  ofs[ofs_count].room = player.Room;
  ofs[ofs_count].frame = frame;
  ofs[ofs_count].oID = this.ID;
  ofs[ofs_count].clip = clip;
  ofs_count++;
}

void PlayObjectFrameSounds() {
  for (int i = 0; i < ofs_count; i++) {
    if (ofs[i].room == player.Room && object[ofs[i].oID].Frame == ofs[i].frame) {
      if (ofs[i].channel == null || !ofs[i].channel.IsPlaying) ofs[i].channel = ofs[i].clip.Play();
    }
  }
}

Now add PlayObjectFrameSounds(); to the global repeatedly_execute.

In your room_Load, use

Code: ags

  oEgyptRun1.AddFrameSound(127, aGUNSHOT1);

[Gal Shemesh]

Oh, of course! I'm using @Khris's help with custom speech function which use the audio channel method, and wondered why I can't just use the '!= null' here as well. Thanks for the heads up!

I know that the new audio system has been discussed many times in many places around the forum - I spent HOURS last night reading many threads about it. It is still a mystery to me to understand. But only now note that people use different names when using this method: some use 'channel', some simply use 'ch', and you here used 'audio_channel'. So it maybe becomes more clear to me now - can this name be 'anything' we like, so theoretically I can decide in which "channel" name to use for specific audio elements?

[Khris]

What you call your variables/pointers is completely up to you. The important thing is the correct type, in this case AudioChannel*.

[Gal Shemesh]

Thanks @Khris! Yet another one of your custom function. This one looks even more complicated for me to understand than the others, so I'm going to go over it and try to understand how it works line by line now. Thanks a lot!

[Khris]

I'll leave it here as example code for similar stuff, but you can do the same like this:

Code: ags

  ViewFrame* vf = Game.GetViewFrame(EGYPTRUN1, 0, 127); // 0 is the loop
  vf.LinkedAudio = aGUNSHOT1;

[Nahuel]

So then wouldn't be possible to achieve also directly on the view under the Design>Sound then ?

[Khris]

Sure, but Gal explains in the first post why this is cumbersome because his views have so many frames. That's why he wants to do it via script.

[Gal Shemesh]

Sure, but Gal explains in the first post why this is cumbersome because his views have so many frames. That's why he wants to do it via script.

Exactly. You were ahead of me relating to this as I was composing a reply to @Nahuel.

Speaking on the View editor for setting audio in frames, I also posted another feature request today regarding it here, as currently when seeking for a specific frame in the view editor preview, you have to manually seek for this frame either in the row of frames or within the dropdown list of frames in the properties panel, which both perform quite slow when you have great amount of frames.

[Gal Shemesh]

Is there a way to also use characters 'speech' files to play on a specific frame in an animation?
Currently the conversation I have is a close-up photographed footage of a conversation between two characters (not the actual characters during play mode but like a cut-scene video). And I'm willing to use my custom 'Say' function during the conversation animation, which shows their text at the bottom of the screen.

I was trying to do it this way, but of course it doesn't work at I used the 'Game.PlayVoiceClip' which plays the voice clip - not set it. Is there a way to set an assignment of a voice clip to a specific frame?

Code: ags

oConversation.playAudioClipOnFrame(0, 2, Game.PlayVoiceClip(cMei, 1);

Thanks

[Khris]

You need a second function and call it like this:

Code: ags

oConversation.playVoiceClipOnFrame(0, 2, cMei, 1);

Also note that in general, if

Code: ags

oConversation.playAudioClipOnFrame(0, 2, Game.PlayVoiceClip(cMei, 1));

worked, it still wouldn't do what you want.

You wouldn't schedule the Game.PlayVoiceClip call at all, rather it would run as soon as AGS hits that line. Next it would call oConversation.playAudioClipOnFrame and pass the *return value* of the Game.PlayVoiceClip call as third parameter.

There are scripting languages where stuff like this is possible, but for this you need the ability to pass functions as parameters.
Then you could do

Code: ags

oConversation.runFunctionOnFrame(0, 2, function () { Game.PlayVoiceClip(cMei, 1); });

Unfortunately, this fantastic technique is not supported (yet?).

[Gal Shemesh]

Thanks, @Khris! Much appreciated. I guess it makes things more complicated for me... Well, maybe it will be supported in AGS in the future.

EDIT:
Meanwhile, I ended up adding the specific speech files used in the cut-scenes directly into AGS, and now they can be assigned to 'view' frames same as I can do with SFX under the 'room_Load()' function - not the best approach I was after, but it sort of works. This is the custom function I'm using:

Code: ags

void playAudioClipOnFrame(this Object*, int loop, int frame, AudioClip* sound)
{
  ViewFrame* vf = Game.GetViewFrame(this.View, loop, frame);
  vf.LinkedAudio = sound;
}

Though, I eventuaully found that assigning speech files to be played in specific frames with the above method for 'cut-scenes' in particular became into a tedious process, as there are quite some dialogues in the 'cut-scenes' and it requires to actually import all the relevent speech audio files into the editor - which isn't ideal. So instead, I found that if I use a simple 'if' statement to check for a specific 'cut-scene' frame, I can actually just use the regular '.SayAt' function under 'room_RepExec()', which pulls the speech directly from the external 'Speech' folder.

Here's an example of a cut-scene, which is basically a background and an object playing a cropped animation view of the moving area of the scene:



Here's the custom '.sayNew' function code of the above screenshot:

Code: ags

function room_RepExec()
{
  if (oZepConversation.Frame == 65)
  {
    cRob.sayNew(6, "I'm lucky I made it out of Egypt.");
  }
}

And this is the actual code of the custom speech function:

Code: ags

void sayNew(this Character*, int cue, String text)
{
  String cueString = String.Format("&%d", cue);
  {
    if (Speech.VoiceMode == eSpeechVoiceAndText && IsSpeechVoxAvailable())
    {
      AudioChannel* ch = Game.PlayVoiceClip(this, cue);
      this.SayAt(Game.Camera.Width /2, Game.Camera.Height + 15, -1, text);
      if (ch != null)
      {
        ch.Stop();
      }
    }
    else if (Speech.VoiceMode == eSpeechVoiceOnly && IsSpeechVoxAvailable())
    {
      this.Say(cueString);
    }      
    else if (Speech.VoiceMode == eSpeechTextOnly)
    {
      this.SayAt(Game.Camera.Width /2, Game.Camera.Height + 15, -1, text);
    }
  }
}

But there are 2 issues that I face now, which I'm trying to solve by my own for quite some time using the manual and forum search, however so far I haven't found a solution - perhaps you or someone else can assist on this one:

1. When there's a speech line that is triggered to be said before that a previous line's text has been removed from the screen, the new line 'slips through the cracks' and not said at all, since the '.Say' function doesn't return immediately - so basically the frame check that suppose to play the next speech line is passed before the previous speech text has been removed from the screen, missing the only chance it has to play it. For example:

Code: ags

if (oZepConversation.Frame == 65)
{
  cRob.sayNew(6, "I'm lucky I made it out of Egypt.");
}
if (oZepConversation.Frame == 83)
{
  cMei.sayNew(4, "What did they want?"); // <- this line 'slips through the cracks'
}

So I'm trying to figuring out how to make my custom '.sayNew' function to actually 'interrupt' any previous speech text and to remove it from the screen immediately when it is called (like when the players can manually skip text lines, only upon call), so the next '.sayNew' could be triggered without interference. Currently it doesn't return to the code until the speech text is automatically removed.

2. Players are able to skip speech lines, which I want during the game, but not in 'video like' cut-scenes animation like I have above - as skipping the text is irrelevant since the players will then see a 'muted' cut-scene animation until the next frame that triggers the next speech line is reached. So I'm trying to prevent skipping the text lines specifically during cut-scenes.

I've found that playing the speech audio separately and using the '.SayBackground' function to display the text is almost best to bypass both issues above, which both prevents skipping and return to the code immediately. So any triggers for next speech lines are successfully checked on time:

Code: ags

if (oZepConversation.Frame == 65)
{
  Game.PlayVoiceClip(cRob, 6);
  cRob.SayBackground("I'm lucky I made it out of Egypt.");
}
if (oZepConversation.Frame == 83)
{
  Game.PlayVoiceClip(cMei, 4);
  cMei.SayBackground("What did they want?");
}

However, it has its flaws:

1. Text cannot be positioned anywhere I like, same as I can do with '.SayAt', which also allows me to set '-1' for the text width for having a nice one-line of text. for this; I found that I can bypass this by positioning the characters at the bottom center of the screen where I want the text to be shown and to make them transparent. However, there is no text 'width' option to set in this function, and so long text lines automatically break into new lines, which in most cases put a 'single' or 'two words' on a second line because it ran out of space, which looks odd.

2. If 2 characters' speech lines are triggered close to each other in time, an overlapping of the speech text may occur; I found that I can bypass this by passing a 'Character.SayBackground("");' for the previous character that spoke, just before the speech line of the next character, which wipes the previous character's text and prevents this overlap.

3. It requires 2 separate lines for each cue; for playing the speech and for showing its text - this can be easily combined in a custom function, but I'll first need to solve the 2 issues above of positioning the '.SayBackground' text in specific X and Y coordinates, and to have its width as '-1' to prevent oddly line breaks.

Would appreciate some help. Thanks