48. Scenes, Menus and Sound¶
What surrounds the rules: the screens a game shows one after another, the menus every game has, the high-score dialogs, menus drawn into the picture of the game, sound, and voices that speak. These parts make a set of rules feel like a finished game, and they are the same in every game, which is why they are in the engine.
Screens: Scene and SceneStack¶
Package scene · introduced in 20. Paddle; the overlay in 21. Bricks · source: Scene, SceneStack
What it is for. The screens of a real-time game: a title, the match, a pause screen, the end. A Scene advances in fixed steps (update(seconds)) and draws itself in logical coordinates (render(g, alpha)), with an optional renderOverlay for what lies on top of the world, such as the score. Both run on the game loop thread.
How it works. A SceneStack is itself a scene, and the game loop's update and render functions call it. Only the top scene is updated, but all are drawn from the bottom up: a pause screen pushed onto the match freezes it and still shows it underneath. The overlays of all scenes come after all worlds, so a pause text lies over the match's score.
public final class SceneStack implements Scene {
private final List<Scene> scenes = new ArrayList<>();
/** Puts a scene on top; the one below stops moving. */
public void push(Scene scene) {
scenes.add(Objects.requireNonNull(scene, "scene"));
}
/** Removes the top scene; the one below moves again. */
public void pop() {
if (!scenes.isEmpty()) scenes.removeLast();
}
/** Replaces all scenes by one, e.g. from the title to the match. */
public void replace(Scene scene) {
scenes.clear();
push(scene);
}
public Scene top() {
return scenes.isEmpty() ? null : scenes.getLast();
}
public int size() {
return scenes.size();
}
@Override
public void update(double seconds) {
if (!scenes.isEmpty()) scenes.getLast().update(seconds);
}
@Override
public void render(Graphics2D g, double alpha) {
// A copy: a scene may push or pop while the frame is drawn.
for (Scene scene : List.copyOf(scenes)) scene.render(g, alpha);
}
/** The overlays of all scenes, bottom first, after all worlds: a pause text lies over the match's score. */
@Override
public void renderOverlay(Graphics2D g, double alpha) {
for (Scene scene : List.copyOf(scenes)) scene.renderOverlay(g, alpha);
}
}
| Method | What it does |
|---|---|
push(scene) |
On top; the one below stops moving: pause, a menu over the game |
pop() |
Removes the top; the one below moves again |
replace(scene) |
All scenes replaced by one: from the title to the match, from the match to its end |
top(), size() |
The scene that moves |
A scene is the right place for what belongs to one screen only: the title's demo game, the end screen's result. Changes that come from Swing, such as New Game in the menu, reach the stack through GameLoop.post (chapter 41).
The standard menus: GameMenus and GameTexts¶
Package swing · introduced in 10. Mines; undo and redo in 11. Solitaire, full screen in 20. Paddle, graphics and frame times in 21. Bricks · source: GameMenus, GameTexts
What it is for. The menu entries every game has, with the same texts, mnemonics and keys in all of them, in the five languages of amadeOS (English, German, French, Spanish and Italian). The game adds its own entries between them.
| Method | Entry | Key |
|---|---|---|
game(), help() |
The menus Game and Help | |
newGame(action) |
New Game | F2 |
undo(action), redo(action) |
Undo, Redo; enable them while there is something to undo | Ctrl+Z, Ctrl+Y |
fullScreen(action) |
Full Screen, usually FullScreen::toggle |
F11 |
graphics(selected, action) |
The submenu Graphics, one entry per RenderMode |
|
frameTimes(action) |
Frame Times, ticked while they show | F12 |
highScores(action) |
High Scores... | |
exit(action) |
Exit | Ctrl+Q |
about(title, action) |
About <title> |
Every item has a component name (game-new, game-full-screen, …), so tests of the window find it. GameTexts.load() gives the engine's texts in the desktop's language; GameTexts.BUNDLE is the bundle's name for amade-commons' TranslationCheck, which a game's tests run on its own texts and the engine's together, so two entries of one menu never share a mnemonic (chapter 2).
High-score dialogs: HighScoreDialog¶
Package swing · introduced in 10. Mines · source: HighScoreDialog
HighScoreDialog.show(parent, scores, tables, format, selected, highlight) shows the high scores, one tab per table, with a new entry highlighted and a Reset button that asks first; format turns a value into text, 42 s or 1,250. HighScoreDialog.askName(parent, message, playerName) asks for the name after a good result, offering the one typed last time, and remembers the new one. Both are modal Swing dialogs and must be called on the event thread. A real-time game that runs in full screen shows its own end screen as a scene instead, with the place in the table drawn into the picture.
Menus over the game: OverlayMenu¶
Package menu · introduced in 30. Towers · source: OverlayMenu
What it is for. A menu inside the game's picture that opens beside the place it belongs to, such as the tile the player clicked, and offers a few things to choose: the towers to build there and what they cost. Each Item has a value, an optional icon, a label, an optional detail on the right (a price) and whether it can be chosen; a tower the player cannot pay shows, but greyed.
When to choose it. For commands given on the map. A Swing popup cannot do it: the canvas covers Swing's lightweight popups, a heavyweight one is a window of its own over a game in full screen, and neither looks like the game. Settings and everything else outside the play stay in the Swing menu bar.
How it works. The menu is arithmetic first: where it lies, which row is under the mouse, what a click or a key chooses. open(title, items, anchorX, anchorY, anchorWidth, anchorHeight) places it to the right of the anchor, vertically centred, to its left when there is no room, and inside the bounds given to the constructor, the part of the screen it may cover; setBounds changes them when the screen widens. The game hands it the mouse and the keys in its update step, so the menu needs no listeners and its tests no screen:
| Method | What it does |
|---|---|
hover(x, y) |
Highlights the row under the mouse |
click(x, y) |
The value of a row that can be chosen; outside the menu, it closes without a choice; null when nothing was chosen |
key(number) |
The keys 1, 2, 3 for the first rows |
move(+1 or -1), chooseHighlighted() |
The arrow keys and Enter |
contains(x, y) |
A click on the menu belongs to the menu, not to the map below |
update(seconds), render(g, style) |
It opens in a tenth of a second, growing and fading in; drawn in the overlay with a Style, Style.DEFAULT being dark wood and brass |
/**
* Opens the menu beside the box {@code anchorX}, {@code anchorY}, {@code anchorWidth} ×
* {@code anchorHeight}, the thing it is about: to its right, vertically centred on it; to its left when
* there is no room on the right; and moved up or down as far as needed to stay inside the bounds. The
* first row that can be chosen is highlighted, so Enter chooses it.
*/
public void open(String title, List<Item<T>> items, double anchorX, double anchorY,
double anchorWidth, double anchorHeight) {
if (items.isEmpty()) throw new IllegalArgumentException("a menu needs something to choose");
this.title = title;
this.items = List.copyOf(items);
double height = height();
x = anchorX + anchorWidth + GAP;
if (x + width > boundsRight) x = anchorX - GAP - width;
x = Math.clamp(x, boundsLeft, Math.max(boundsLeft, boundsRight - width));
y = anchorY + anchorHeight / 2 - height / 2;
y = Math.clamp(y, boundsTop, Math.max(boundsTop, boundsBottom - height));
highlighted = -1;
move(1);
openness = 0;
open = true;
}
Test: chosen by click and by key
@Test
void theMouseAndTheKeysChooseWhatCanBeChosen() {
menu.open("Build", towers, 100, 200, 20, 20);
double left = menu.x() + 20;
double row = menu.y() + 16 + 13; // the middle of the first row
menu.hover(left, row + 26);
assertEquals(1, menu.highlighted(), "the archer under the mouse");
assertNull(menu.click(left, row + 52), "the frost tower is too dear");
assertTrue(menu.isOpen());
assertNull(menu.click(left, menu.y() + 5), "on the title: nothing");
assertEquals("archer", menu.click(left, row + 26));
assertFalse(menu.isOpen(), "a choice closes the menu");
menu.open("Build", towers, 100, 200, 20, 20);
assertNull(menu.key(3), "3 is the frost tower");
assertEquals("cannon", menu.key(1));
menu.open("Build", towers, 100, 200, 20, 20);
assertEquals(0, menu.highlighted(), "the first that can be chosen");
menu.move(-1);
assertEquals(1, menu.highlighted(), "up from the first wraps round, past the frost tower");
assertEquals("archer", menu.chooseHighlighted());
menu.open("Build", towers, 100, 200, 20, 20);
assertNull(menu.click(10, 390), "outside");
assertFalse(menu.isOpen(), "a click outside closes it");
}
Sound effects: Sound¶
Package audio · introduced in 20. Paddle · source: Sound
What it is for. A short sound as raw samples: 44,100 numbers per second, each the position of the loudspeaker's membrane between -32768 and 32767. That is all a sound card plays. The effects of the early arcade games were not recordings but tones the machine made, and so are the sounds of this book.
How it works. Sound.tone(wave, fromHz, toHz, seconds, volume) makes a tone that slides from one pitch to another, with a fade of 5 ms in and out so the speaker does not click. The wave is SQUARE (the hard blip of the arcade), TRIANGLE (softer) or NOISE (crashes and explosions; the same noise every time, from a fixed seed). then(next) joins sounds into a little melody, Sound.mix(sounds...) plays several at once as one sound, Sound.of(samples) takes samples from anywhere (chapter 20).
/**
* A tone sliding from {@code fromHz} to {@code toHz} within {@code seconds}, at {@code volume} 0 to 1.
* Noise ignores the pitch.
*/
public static Sound tone(Wave wave, double fromHz, double toHz, double seconds, double volume) {
int n = (int) Math.round(seconds * RATE);
short[] samples = new short[n];
int fade = Math.min(n / 4, RATE / 200); // 5 ms in and out against clicks
double phase = 0; // where in one vibration we are, 0 to 1
long noise = 0x2545F4914F6CDD1DL; // a fixed seed: the same sound every time
for (int i = 0; i < n; i++) {
double hz = fromHz + (toHz - fromHz) * i / n;
phase = (phase + hz / RATE) % 1.0;
double value = switch (wave) {
case SQUARE -> phase < 0.5 ? 1 : -1;
case TRIANGLE -> 4 * Math.abs(phase - 0.5) - 1;
case NOISE -> {
noise ^= noise << 13;
noise ^= noise >>> 7;
noise ^= noise << 17;
yield (noise >>> 11) / (double) (1L << 53) * 2 - 1;
}
};
double envelope = Math.min(1, Math.min(i, n - 1 - i) / (double) Math.max(1, fade));
samples[i] = (short) Math.round(value * envelope * volume * Short.MAX_VALUE);
}
return new Sound(samples);
}
Make the sounds once, at the start, in a sound bank of the game: a sound is a few kilobytes of samples, and making it costs more than playing it.
Tunes as text: Tune¶
Package audio · introduced in 24. Swarm · source: Tune
A melody written as text in the Music Macro Language of the home computers of the 1980s, one voice per string: Tune.voice(Sound.Wave.SQUARE, 0.3, "t140 o4 l8 c e g > c"). Notes c to b with +, # or - and an optional length (4 a quarter, a dot half as long again), r a rest, l the default length, o the octave (in octave 4, a is 440 Hz), > and < an octave up and down, t the tempo in quarter notes per minute; spaces and bars | are only for the eye. Each note sounds a little shorter than its length, so repeated notes are heard apart, and notes are placed by their beat, so voices of different rhythm stay together. Sound.mix plays melody and bass as one sound.
@Test
void notesHaveTheirPitchAndLength() {
assertEquals(440, Tune.frequency(4, 9), 1e-9); // A in octave 4
assertEquals(261.63, Tune.frequency(4, 0), 0.01); // middle C
assertEquals(880, Tune.frequency(5, 9), 1e-9); // an octave up: twice the frequency
// Tempo 60: a quarter note is a second. A whole A, then a half rest, then a dotted quarter C an octave up.
Sound tune = Tune.voice(Sound.Wave.SQUARE, 0.5, "t60 o4 a1 r2 > c4.");
assertEquals((4 + 2 + 1.5) * Sound.RATE, tune.length());
assertEquals(440, crossings(tune, 0, Sound.RATE), 2, "440 Hz");
assertEquals(0, tune.sample(5 * Sound.RATE), "the rest");
assertTrue(crossings(tune, 6 * Sound.RATE, (int) (6.5 * Sound.RATE)) > 120, "C5, 523 Hz");
}
Playing: Mixer¶
Package audio · introduced in 20. Paddle; music in 24. Swarm · source: Mixer
What it is for. Sound effects, several at once: a brick breaks while the ball hits the wall, and both must be heard together without either waiting for the other. And one piece of music, once or again and again, on a voice of its own that no effect can push out.
How it works. The mixer opens one line to the sound card and feeds it from a thread of its own, block by block of 512 samples (11.6 ms, the longest a new sound waits). For every block it adds up the samples of all sounds that are playing, which is all mixing is, and clips the sum to the range of a sample. At most 16 effects play at once; a new one replaces the oldest. play(sound) only puts the sound in a queue and returns, from any thread: the game loop never waits for the sound card.
/** Fills {@code block} with the sum of all playing sounds; on the mixer thread, or in tests. */
void mix(short[] block) {
for (Sound s; (s = requests.poll()) != null; ) {
if (voices.size() == VOICES) voices.removeFirst();
voices.add(new Voice(s));
}
MusicRequest change = musicRequest.getAndSet(null);
if (change != null) {
music = change.sound() == null ? null : new Voice(change.sound());
musicLoops = change.loop();
}
for (int i = 0; i < block.length; i++) {
int sum = 0;
for (Voice v : voices) {
if (v.position < v.sound.length()) sum += v.sound.sample(v.position++);
}
sum += music();
// Two loud sounds together exceed what 16 bits can hold: clip instead of wrapping around.
block[i] = (short) Math.clamp(sum, Short.MIN_VALUE, Short.MAX_VALUE);
}
voices.removeIf(v -> v.position >= v.sound.length());
}
| Method | What it does |
|---|---|
start(), close(), running() |
Opens and closes the sound card; without one, the mixer stays silent and the game plays on |
play(sound) |
An effect, as soon as possible |
playMusic(sound, loop), stopMusic() |
The music, from its start; once or in a loop |
setMuted(muted) |
New effects are dropped and the music turns silent but goes on, so it is where it would be when sound comes back |
Behind the scenes: a hiss nobody hears
The mixer adds a hiss of one or two steps out of 32,768 to every block, far below hearing, so the line never carries pure silence. Some wireless headsets switch off after a few seconds of zeros and take longer to wake up again than a game can wait: the first sounds of a round would be lost (appendix).
Speech: Speech and Phoneme¶
Package audio · introduced in 28. Hover Cab · source: Speech, Phoneme
What it is for. Characters who talk, without recordings: "Hey, cab!", "Pad 3, please!", in any language and in a voice per character. Speech.say(voice, phrase) returns a Sound to play with the Mixer.
When to choose it. For short calls and announcements where a voice adds life and recordings would be too many (characters × phrases × languages) or not wanted. It sounds like the talking machines of the early 1980s, which is its charm; for long dialogue, recordings are better.
How it works. A formant synthesizer: a model of the voice as a source and a filter. The source is the vocal folds, puffs of air at the voice's pitch, plus hiss for an s and breath for an h; the filter is the mouth, five resonators (Dennis Klatt's two-pole filters of 1980) tuned to the formants, the frequencies at which the mouth rings. Each Phoneme says where its formants are and how the voice makes it; between phonemes the formants glide through anchors, as a tongue moves, and the sources fade instead of switching. The pitch follows a tune: it sinks over the phrase, rises on stressed vowels, starts high in a call (!) and rises at the end of a question (?) (chapter 28).
public record Voice(double pitch, double formants, double pace, double breath) {
public static final Voice MAN = new Voice(112, 1.0, 1.0, 0.04);
public static final Voice WOMAN = new Voice(205, 1.15, 1.0, 0.08);
public static final Voice CHILD = new Voice(280, 1.28, 0.95, 0.06);
public static final Voice OLD_MAN = new Voice(92, 0.97, 0.85, 0.16);
public static final Voice ROBOT = new Voice(100, 1.05, 1.0, 0);
public Voice {
if (pitch < 40 || pitch > 600) throw new IllegalArgumentException("pitch out of range: " + pitch);
if (formants < 0.7 || formants > 1.5) throw new IllegalArgumentException("formants out of range: " + formants);
if (pace <= 0) throw new IllegalArgumentException("pace must be positive: " + pace);
}
/** The same voice at another pitch: to tell two passengers apart who are both women. */
public Voice withPitch(double hz) {
return new Voice(hz, formants, pace, breath);
}
}
A phrase is written in the ARPAbet, phonemes separated by spaces: hh ey1 , k ae1 b !. A 1 stresses a vowel, , and . are short and long pauses, / separates words only for the eye. An unknown phoneme is an error that names it. The sounds were made for English but give a good accent in other languages; keep the phonemes next to the words in the message files, so translators write both.
Speaking a phrase of a second takes 10 to 20 ms, more than a step of the game. Make the phrases ahead, at the start or on a thread of their own, and keep them (chapter 28).
| Method | What it does |
|---|---|
Speech.say(voice, phrase), say(voice, phrase, volume) |
The phrase as a Sound, its loudest moment at volume |
Voice.MAN, WOMAN, CHILD, OLD_MAN, ROBOT |
Voices: pitch in Hz, a factor on the formants (the size of the mouth), pace, breath |
voice.withPitch(hz) |
The same voice higher or lower, to tell two characters apart |
Phoneme.named("ae") |
A phoneme by its ARPAbet name |
Test: vowels by their formants, voices by their pitch
@Test
void vowelsAreToldApartByTheirFormants() {
Sound ee = Speech.say(Speech.Voice.MAN, "iy");
Sound ah = Speech.say(Speech.Voice.MAN, "aa");
int a = ee.length() / 4, b = ee.length() * 3 / 4;
// "ee" has its second formant high, near 2300 Hz; "ah" has it low, near 1100 Hz
assertTrue(energy(ee, a, b, 2290) > 10 * energy(ee, a, b, 1100), "ee rings high");
int c = ah.length() / 4, d = ah.length() * 3 / 4;
assertTrue(energy(ah, c, d, 1090) > 10 * energy(ah, c, d, 2290), "ah rings low");
}
@Test
void aWomansVoiceIsHigherThanAMans() {
Sound man = Speech.say(Speech.Voice.MAN, "aa");
Sound woman = Speech.say(Speech.Voice.WOMAN, "aa");
double low = pitch(man, man.length() / 3, man.length() / 3 + 2000);
double high = pitch(woman, woman.length() / 3, woman.length() / 3 + 2000);
assertEquals(1.08 * 112, low, 15, "about the voice's pitch, a little raised at the start of a phrase");
assertTrue(high > 1.6 * low, high + " Hz against " + low + " Hz");
}