43. Input

A game that waits for the player reacts to events: a click arrives, the presenter plays the move. A real-time game asks instead: 120 times a second the rules want to know which keys are held, where the mouse is, whether a button went down since the last look. The input classes of the engine turn Swing's events into answers to those questions.

One pattern for every device

All input classes work the same way, because two threads meet in them:

1. Swing reports. Keyboard.attach(canvas) and Mouse.attach(canvas) add listeners to the canvas; the gamepad backends report from a thread of their own or when asked. What they report is collected.
2. The game loop polls. At the start of each update step the game calls poll() on each device it uses. The device takes over what was collected, and all questions of that step get the same answers.
3. The rules ask. isDown (held now), wasPressed (went down between the last two polls, however short the press was), and for the mouse wasReleased and the position.

The second point matters more than it looks. A key pressed and released between two steps is never "down" when the rules look, but wasPressed still sees it. And a step that asks twice cannot get two different answers because an event arrived in between.

Device	Poll	Introduced in
Keyboard	keyboard.poll()	20. Paddle
Mouse	mouse.poll()	21. Bricks
Gamepads	gamepads.poll()	26. Blast

The keyboard: KeyState and Keyboard

Package input · introduced in 20. Paddle · source: KeyState, Keyboard

What it is for. Which keys are held, and which went down since the last look. KeyState is the bookkeeping, without AWT: key codes and nanosecond times come in, so tests can play any sequence. Keyboard connects it to a component and makes it safe between the two threads.

How it works. Real keyboards bring two problems, and KeyState handles both:

• Key repeat on X11. A held key makes X11 send release, press, release, press, a few milliseconds apart. A game that believed every release would see a stuttering paddle. A release is therefore only believed when no press of the same key follows within 25 ms (REPEAT_WINDOW).
• Short taps. A press is remembered until the next poll, so wasPressed sees a tap that ended before the rules looked.

amade-game-engine · input/KeyState.java

public final class KeyState {
    /** How long a release waits for the press that would make it a key repeat. */
    public static final long REPEAT_WINDOW = 25_000_000;

    private final Set<Integer> down = new HashSet<>();
    private final Map<Integer, Long> releasing = new HashMap<>();
    private final Set<Integer> pressedSincePoll = new HashSet<>();
    private Set<Integer> pressed = Set.of();

    public void pressed(int key, long now) {
        if (releasing.remove(key) != null) return;    // release and press of a key repeat: still held
        if (down.add(key)) pressedSincePoll.add(key);  // a new press (repeated presses are ignored)
    }

    public void released(int key, long now) {
        if (down.contains(key)) releasing.put(key, now);
    }

    /** Takes the presses since the last poll and lets releases older than the window take effect. */
    public void poll(long now) {
        for (Iterator<Map.Entry<Integer, Long>> it = releasing.entrySet().iterator(); it.hasNext(); ) {
            Map.Entry<Integer, Long> release = it.next();
            if (now - release.getValue() >= REPEAT_WINDOW) {
                down.remove(release.getKey());
                it.remove();
            }
        }
        pressed = Set.copyOf(pressedSincePoll);
        pressedSincePoll.clear();
    }

    /** Forgets all held keys at once, e.g. when the window loses the focus and their releases never come. */
    public void releaseAll() {
        down.clear();
        releasing.clear();
    }

    /** True while the key is held, as of the last poll and the events since. */
    public boolean isDown(int key) {
        return down.contains(key);
    }

    /** True if the key went down between the last two polls, however short the press was. */
    public boolean wasPressed(int key) {
        return pressed.contains(key);
    }
}

Keyboard.attach(component) makes the component focusable, turns off focus traversal (Tab is a key like any other in a game) and forgets all held keys when the window loses the focus, because their releases would never arrive.

Pitfall: two taps of the same key in a test

Because a press within 25 ms of a release counts as key repeat, a test that taps the same key twice must let a few steps pass between the taps, or the second tap is taken for the first one still held.

Test: key repeat is not a release

amade-game-engine · test/input/KeyStateTest.java

    @Test
    void x11KeyRepeatDoesNotLookLikeReleases() {
        KeyState keys = new KeyState();
        keys.pressed(UP, 0);
        keys.poll(1 * MS);
        assertTrue(keys.wasPressed(UP));
        // Holding the key: X11 sends release and press again, 1 ms apart, every 30 ms.
        for (long t = 500 * MS; t < 1000 * MS; t += 30 * MS) {
            keys.released(UP, t);
            keys.pressed(UP, t + MS);
            keys.poll(t + 10 * MS);
            assertTrue(keys.isDown(UP));
            assertFalse(keys.wasPressed(UP), "a repeat is not a new press");
        }
        keys.released(UP, 2000 * MS);                     // really let go
        keys.poll(2010 * MS);
        assertTrue(keys.isDown(UP), "not yet: a press may still follow");
        keys.poll(2030 * MS);
        assertFalse(keys.isDown(UP));
    }

Actions instead of keys: Controls

Package input · introduced in 20. Paddle, pad buttons in 26. Blast · source: Controls

What it is for. The rules ask "does the right player move up?", never "is the up arrow down?". Controls maps the actions of a game, an enum, to keys and gamepad buttons; several keys and buttons can trigger one action.

When to choose it. Always, in the rules. The keyboard itself is only asked directly for keys that are no action of the game, such as the number keys that recall a group in Sands. With Controls, a second player, a gamepad or keys from the settings need no change in the rules.

amade-game-engine · input/Controls.java

public final class Controls<A extends Enum<A>> {
    private record PadButton(Gamepad pad, Gamepad.Button button) {
    }

    private final Map<A, List<Integer>> keys;
    private final Map<A, List<PadButton>> buttons;
    private final Keyboard keyboard;

    public Controls(Class<A> actions, Keyboard keyboard) {
        this.keys = new EnumMap<>(actions);
        this.buttons = new EnumMap<>(actions);
        this.keyboard = keyboard;
    }

    /** Adds keys that trigger an action. */
    public Controls<A> bind(A action, int... keyCodes) {
        List<Integer> list = keys.computeIfAbsent(action, a -> new ArrayList<>());
        for (int key : keyCodes) list.add(key);
        return this;
    }

    /** Adds buttons of a gamepad that trigger an action. */
    public Controls<A> bind(A action, Gamepad pad, Gamepad.Button... padButtons) {
        List<PadButton> list = buttons.computeIfAbsent(action, a -> new ArrayList<>());
        for (Gamepad.Button button : padButtons) list.add(new PadButton(pad, button));
        return this;
    }

    /** Replaces the keys of an action. */
    public Controls<A> rebind(A action, int... keyCodes) {
        keys.remove(action);
        return bind(action, keyCodes);
    }

    public boolean isDown(A action) {
        for (int key : keys.getOrDefault(action, List.of())) if (keyboard.isDown(key)) return true;
        for (PadButton b : buttons.getOrDefault(action, List.of())) if (b.pad().isDown(b.button())) return true;
        return false;
    }

    public boolean wasPressed(A action) {
        for (int key : keys.getOrDefault(action, List.of())) if (keyboard.wasPressed(key)) return true;
        for (PadButton b : buttons.getOrDefault(action, List.of())) if (b.pad().wasPressed(b.button())) return true;
        return false;
    }

    /** -1 while only the first action is held, +1 while only the second, else 0: e.g. up and down. */
    public int axis(A negative, A positive) {
        return (isDown(positive) ? 1 : 0) - (isDown(negative) ? 1 : 0);
    }
}

Method	What it does
bind(action, keyCodes...)	Adds keys, KeyEvent.VK_UP and the like
bind(action, pad, buttons...)	Adds buttons of one gamepad
rebind(action, keyCodes...)	Replaces the keys, e.g. from the settings
isDown(action), wasPressed(action)	Any of its keys or buttons
axis(negative, positive)	-1, 0 or +1: left and right, up and down

Each player gets a Controls of their own: two players on one keyboard in Paddle, a seat per player with keys or a pad in Blast (chapter 26).

The mouse: Mouse

Package input · introduced in 21. Bricks; held buttons in 23. Rocks, release and inside in 30. Towers, the wheel in 33. Town · source: Mouse

What it is for. The mouse of a game, like the keyboard: Swing reports, the loop polls and asks. Positions are in the game's logical coordinates, so a game never sees the size of the window, the black bars or the render mode.

How it works. attach(canvas) listens to the canvas and converts every position with canvas.toLogical, which places points exactly as the current render mode places the picture (chapter 41). A press also asks for the keyboard focus, so a click into the game gives it the keys. Buttons are bits of an int, one per button, for what is held now and for what went down and up since the last poll.

Method	What it does
x(), y()	The last position, logical; NaN until the mouse has been over the canvas
wasMoved()	It moved between the last two polls
wasPressed(button), wasReleased(button)	MouseEvent.BUTTON1 and the others; a release ends a drag
isDown(button)	Held at the last poll: thrust while the right button is held
isInside()	Over the canvas: a game that scrolls at the edge stops when the mouse leaves
wheel()	The notches the wheel turned between the last two polls: positive towards the player, 0 if not at all

A position outside the picture, on a black bar, lies outside 0 … logical size; check it before you use it. A map larger than the screen needs one more conversion, from the screen to the world: Camera.worldX. A click or a drag on that map becomes a selection with BoxSelect.

Gamepads: Gamepads and Gamepad

Package input · introduced in 26. Blast · source: Gamepads, Gamepad, LinuxJoysticks, XInput

What it is for. Up to four gamepads, in slots filled in the order the pads are plugged in, plugged in and out while the game runs. Each Gamepad answers like the keyboard: isDown(Button.A), wasPressed(Button.START), anyPressed() (to join a game), and the left stick as x() and y(), -1 to 1. The buttons are named as on the pads of the Xbox, which most pads copy: A, B, X, Y, LB, RB, BACK, START and the d-pad UP, DOWN, LEFT, RIGHT.

When to choose it. In every game that more than one person plays, or that is played from the sofa. Bind the buttons in Controls, next to the keys, and the rules never know which one a player uses. Without a pad, or on a system without a backend, the slots stay empty and the game is played with the keyboard.

How it works. Java has no gamepads of its own, so Gamepads.open() picks a small backend for the operating system:

• Linux: the kernel's joystick devices /dev/input/js0 and on. Each is a file that delivers eight bytes per event; a thread per pad waits for them, and another looks for new pads every second (chapter 26) Not every joystick device is a pad: laptop accelerometers, the tablet mouse of a virtual machine and some keyboards get one too. Before a device takes a slot, its key and property bitmaps in /sys/class/input/jsN/device are read; a device without pad buttons, or with mouse buttons, touch or the accelerometer property, is skipped and logged once (chapter 26).
• Windows: XInput, the gamepad interface of Windows, a C library called through the Foreign Function and Memory API (java.lang.foreign), without JNI and without a native library of our own. It is asked for all four slots in each poll; an empty slot only once a second, because asking it is slow (chapter 26).

Between the backend's thread and the loop, Gamepad.poll() takes over the state, as the keyboard does. The stick has a round dead zone: no stick rests exactly in the middle, and a pad lying on the table would otherwise walk the player slowly to the left. Pushed past half its way, the stick also counts as the d-pad, so a game that only asks for UP and the like works with either.

amade-game-engine · input/Gamepad.java

public final class Gamepad {
    /** The buttons, named as on an Xbox pad. UP to RIGHT are the d-pad, or the left stick pushed far enough. */
    public enum Button {
        A, B, X, Y, LB, RB, BACK, START, UP, DOWN, LEFT, RIGHT;

        final int bit = 1 << ordinal();
    }

    /** Stick movements smaller than this, as a part of the full way, count as none. */
    public static final double DEAD_ZONE = 0.25;
    /** From here on, the stick also presses the d-pad in its direction. */
    public static final double AS_DPAD = 0.5;

    private final int index;
    // Written by the backend, under the lock.
    private boolean reportedConnected;
    private String reportedName = "";
    private int reportedButtons;
    private int pressedSincePoll;
    private double reportedX;
    private double reportedY;
    // As of the last poll, for the game loop.
    private boolean connected;
    private String name = "";
    private int down;
    private int pressed;
    private double x;
    private double y;

    public Gamepad(int index) {
        this.index = index;
    }

    /** The backend's side: the pad is there, with this name. */
    public synchronized void connect(String name) {
        reportedConnected = true;
        reportedName = name;
    }

    /** The backend's side: the pad is gone; its buttons count as released. */
    public synchronized void disconnect() {
        reportedConnected = false;
        reportedButtons = 0;
        reportedX = 0;
        reportedY = 0;
    }

    /**
     * The backend's side: the buttons held now, as bits of {@link Button}, and the left stick, -1 to 1 on
     * both axes, positive to the right and down, as on the screen.
     */
    public synchronized void report(int buttons, double stickX, double stickY) {
        pressedSincePoll |= buttons & ~reportedButtons;
        reportedButtons = buttons;
        reportedX = stickX;
        reportedY = stickY;
    }

    /** Takes over the state the backend reported; once per update step, on the game loop thread. */
    public synchronized void poll() {
        connected = reportedConnected;
        name = reportedName;
        double[] stick = deadZone(reportedX, reportedY);
        x = stick[0];
        y = stick[1];
        int dpad = 0;
        if (x <= -AS_DPAD) dpad |= Button.LEFT.bit;
        if (x >= AS_DPAD) dpad |= Button.RIGHT.bit;
        if (y <= -AS_DPAD) dpad |= Button.UP.bit;
        if (y >= AS_DPAD) dpad |= Button.DOWN.bit;
        int held = reportedButtons | dpad;
        // Pressed: reported as going down since the last poll, also if released again already; or, for the
        // stick's d-pad, held now and not before.
        pressed = pressedSincePoll | (held & ~down);
        down = held;
        pressedSincePoll = 0;
    }

    /**
     * A round dead zone: inside it the stick is at rest; outside, the way from its edge to the full way is
     * stretched to 0 to 1 again, so the player still gets every speed, and the direction is kept.
     */
    static double[] deadZone(double x, double y) {
        double length = Math.hypot(x, y);
        if (length < DEAD_ZONE) return new double[]{0, 0};
        double scaled = Math.min(1, (length - DEAD_ZONE) / (1 - DEAD_ZONE));
        return new double[]{x / length * scaled, y / length * scaled};
    }

Pitfall: native access

Calling C is a restricted operation in Java. Start a game that opens gamepads with --enable-native-access=ALL-UNNAMED, in its tests, its run task and its packaged launcher, and put Enable-Native-Access: ALL-UNNAMED in its jar's manifest, or Java prints a warning on Windows.

Pitfall: the keyboard must always work

A pad the player does not even know about must never lock them out. Let keys and pads join the same way, and give every screen that waits for a pad, such as "Gamepad 2 is not connected", a way out with the keyboard. Blast leaves such a match with Esc.

Tip: pads in tests

Gamepads.none() gives four empty slots without a backend. A test connects a pad with pad.connect("Test pad"), presses buttons with pad.report(Gamepad.bits(Button.A), 0, 0), and polls, as a backend would.