I made a video by screen-recording with Playwright, and it has no sound at all

This is a translated article. Read the original in Korean ↗

I was working on personaverse, a project where AI characters make Shorts videos. I was building a "VTuber" format that places a character cutout over a background clip. The pipeline loads an HTML page in a headless browser and screen-records the whole page to produce the video. When I played the finished video, it had no sound at all, even though the background clip definitely had audio.

Conclusion

This Playwright recording pipeline captures only the screen, so the original audio has to be added in post-processing . Removing muted from the <video> element would not have helped, because nothing in this setup puts an audio track into the recording.

Measure the delay as cut time in the output - time of the same cut in the source. The 1-second value from raw - scene was too large because it also counted the padding at the end. Comparing the same cuts at three points showed the actual delay was 0.27 seconds. When this measurement was automated in code, the more precise result was 0.25 seconds, and the audio was re-muxed with that value . However, the session has no record of the user listening afterward to confirm the sync.

The following is an example, not the actual encode.py source. raw.webm is the silent recording and clip.mp4 is the source. The 250 is the delay in milliseconds that was measured automatically and applied in this case. Replace it with the value you measure for your own video.

ffmpeg -i raw.webm -i clip.mp4 -filter_complex "[1:a:0]adelay=250:all=1,apad[a]" -map 0:v:0 -map "[a]" -c:v libx264 -c:a aac -shortest output.mp4

The -map options select the recorded video and the first audio stream of the source (docs). adelay delays all channels, and apad together with -shortest fills with silence until the video ends (filter docs).

Situation

The project's render engine is a custom pipeline (record.py). It opens an HTML page containing the character cutout and the background video in headless Chromium and screen-records it with Playwright. encode.py/make.py then post-process the result. The browser was launched with --autoplay-policy=no-user-gesture-required so that the background video would always autoplay .

Symptom

I had iterated the "VTuber" format (character cutout + background clip + speech-bubble captions) up to v5. When I played the finished video, none of the video's sound had been recorded. The clip used as the background clearly had audio, but none of it reached the output .

Cause

record.py records only the screen, so removing muted from the background <video> still doesn't capture any audio .

Fix

1. In engine.js, the clip is reset to 0 seconds when playback starts, so it shares the same starting point as the captions .
2. Drop the raw - scene calculation in make.py. Instead, find the time at which the same scene transition happens in both the source and the recording, and use the difference as the delay. To check by hand, grab a still frame near a suspected point with ffmpeg -ss <시각> -i <영상> -frames:v 1 frame.png (<시각> = timestamp, <영상> = video). Then compare the two videos while narrowing the -ss value until you hit the exact moment of the cut (-ss/-frames:v option docs). Comparing three points this way (1.47->1.73s, 2.47->2.73s, 25.67->25.93s) gave the same 0.27-second offset each time .
3. This comparison was automated in code, so each render finds its own scene transitions and calculates the delay. A more precise measurement on the same case gave 0.25 seconds, and the audio was re-muxed with that value . In another episode, though, the video sat in a small player area and a white background with a static character filled most of the frame. Cut detection on the raw recording failed there, so the offset was measured by hand on the final video and entered manually . encode.py takes this delay in seconds and muxes the audio as AAC. The adelay value in the example is seconds × 1,000 (milliseconds).
4. With the existing tool, adjust it with python video-engine/make.py --scene scene.json --skip-record --clip-audio-delay 0.15, substituting your own scene path and delay. A lower value moves the audio earlier .

Bonus — the lead-time calculation was wrong too

The 1-second delay and the caption match looked like success, but the audio was still late . With a small background video, automatic cut detection also failed, so the delay had to be measured by hand on the final video .

Reporter's Notes — why unmuting doesn't bring the sound back

Park Doeun

The VTuber format for Shorts is almost done: character cutout on a background clip, with speech bubbles on top. But I just played the finished video and there's no sound at all

Jung Bada

Isn't the background video tag set to muted? Just remove that

Jung Haneul

If the recording only captures the screen, though, removing muted might not help. How is the recording done?

Park Doeun

Apparently the recording uses a browser automation tool called Playwright, and it only captures the screen . So removing muted won't help

Jung Bada

Huh... so muted was never the problem

Jung Haneul

Right. If you only record the screen, there's no audio track

Park Dohyun

So do we have to change the recording code to capture audio too? That sounds like a lot of work

Jung Haneul

No need. Let's merge in the original audio during encoding

Park Doeun

But the recording doesn't start right away. It starts after the page loads. If we just add the audio, won't the timing be off?

Jung Bada

Treat the delay from that loading as lead time and shift the audio back by that much. That should line it up

Jung Haneul

What if we subtract the actual scene playback length from the raw recording length? Let's use that difference as the delay

Park Dohyun

So, did it work?

Park Doeun

The sound is in. It's a 1-second delay, and the captions line up at 15 seconds, so it all looks right

Jung Bada

Then we're done. Perfectly synced

Jung Haneul

No, the audio is late. Comparing the same cut times shows about 0.27 seconds. My length-difference calculation also counted the padding at the end

Park Dohyun

Then we should only shift the audio by the output's cut time minus the source's time for the same cut

Re-measure using scene transitions and merge the audio.

Park Doeun

So the "no sound at all" problem is solved for now, right?

Jung Haneul

Yes. We know the recording itself only captures the screen, so unmuting doesn't help

Park Doeun

What about the lag? You said it was re-merged with the auto-measured 0.25 seconds

Jung Haneul

Right. That value was measured from scene transitions, so if it sounds off when we listen, we adjust it then

Park Dohyun

When you make videos by screen recording and there's no sound, check how it's being recorded before you look at playback. If the capture only grabs the screen, unmuting won't bring the sound in.