From Game Streaming to KVM: Reducing Input Latency

I set up Sunshine and Moonlight for game streaming from my Linux server to my Windows desk. It worked well for casual streaming, but games that need fast reactions felt sluggish. Dota 2 in particular had noticeable mouse delay. I also play Dota 2 arcade modes on Europe and Peru servers where the base latency is already 130+ ms, and streaming made it worse. This post covers why I switched to a hardware solution.

The Problem with Game Streaming

Sunshine (host) and Moonlight (client) are solid tools for local game streaming. After configuring NVENC encoding at 50 Mbps with wake-on-demand scripts, the stream looked good. The issue was input latency.

The Latency Stack

Every game streaming solution adds latency at multiple points:

That's 17-36 ms added on top of normal input lag. For casual games this is fine. For high-APM games it's noticeable, and it stacks with any existing server latency. Moonlight stats looked fine when idle, but during active gameplay with spell effects and particle systems the experience was different.

The Frame Pacing Problem

I had a small cheap monitor attached to the Linux server for when I want to use it physically. That monitor runs at 60 Hz. My Windows monitor runs at 165 Hz. Streaming from a 60 Hz source to a 165 Hz display creates uneven frame pacing. Frames arrive at irregular intervals, which causes stuttery mouse movement even when the stream stats look smooth.

Capping the client to 60 Hz would fix the pacing, but then there's no point having a high refresh monitor.

Sunshine Session Issues

Sunshine requires an active display session to capture. My Linux server auto-locks for security. When locked, the display powers off and Sunshine has nothing to stream.

I wrote a wake script that would run via SSH to wake the display and restart Sunshine. It worked sometimes, but it was inconsistent. When it failed, I had to walk to the room where the Linux server was and manually log in to get Sunshine running. That got annoying.

I never fully figured out why it was inconsistent. It might have been Wayland vs X11 sessions, or the D-Bus calls not connecting properly from SSH, or the display manager being the only thing active instead of a logged-in desktop. Sometimes it just worked, sometimes it didn't.

What I Tried

Sunshine/Moonlight Troubleshooting

Before giving up on streaming, I tried a lot of things.

Session/wake issues: I wrote a wake script that would run via SSH to wake the display and restart Sunshine. It worked sometimes, but when it failed I had to physically walk to the Linux server and log in manually. I checked session state with loginctl commands and found the issue was often that only GDM existed on seat0, not a logged-in desktop session. Sunshine could be running but had nothing to capture.

Performance tuning: I looked at Moonlight stream stats and noticed decode times were sometimes high. Windows Task Manager showed GPU video activity in the "Video Codec" graph during streaming. I tried various settings:

• Ensured hardware decoding was enabled
• Used HEVC codec (my RX 5700 XT doesn't do AV1 decode)
• Tested capping FPS to match the 60 Hz host
• Fullscreen mode
• Relative mouse mode for Dota

On the Sunshine side:

• Disabled FEC for wired LAN
• Disabled two-pass NVENC mode
• Kept NVENC on fast preset (P1)
• Made sure it wasn't falling back to software encode

Even with good network stats, it still didn't feel local. The encode/decode pipeline adds latency that's hard to eliminate, and the 60 Hz host to 165 Hz client mismatch caused pacing issues.

HDMI Over Cat6 Extenders

I looked into HDMI-over-Cat6 extenders for direct video. Most affordable ones max out at 60 Hz.

The Solution: DP KVM + USB Extender

I found a hybrid approach:

• Video: Long DisplayPort cable directly (no conversion)
• USB: USB-over-Cat6 extender for keyboard/mouse
• Switching: KVM to switch both between Windows and Linux

This provides local video with no encode/decode latency.

The Hardware

USB 2.0 is sufficient for keyboard, mouse, and USB microphone.

The Topology

The UGREEN KVM is DP 1.4, supporting up to 4K@120Hz or 1440p@165Hz. The DP 2.1 cable runs at DP 1.4 speeds (determined by the KVM), handling the 25-foot run without signal issues.

For USB, the TX (transmit) unit connects to the Linux PC. The RX (receive) unit sits at the desk with powered hub ports for peripherals.

The Result

One button press on the KVM switches video and USB together:

• Full 165 Hz refresh rate
• Zero encode/decode latency
• Native mouse feel
• USB mic works through the USB 2.0 extender

Takeaways

Streaming adds unavoidable latency. Sunshine/Moonlight is good for casual gaming, playing from another room, or remote access. For high-APM games, the encode/decode pipeline adds delay that's hard to eliminate.

Direct video eliminates the problem. Removing encode/decode is the only way to get local responsiveness.

Long DP cables work. Rather than a DP-over-Cat6 extender, a quality DP cable handles 25+ feet.

The Sunshine setup still exists for couch gaming. For games that need fast reactions, I use the KVM.