Plan 9: Audio-Video Pipeline

Video pipeline in action

Two important points about videos:

  1. A video is a series of images.
  2. RGB format is used for display. YUV format is used for storage and transmission. 
Recording 
────────────────────────────────────────────────────────

[Video Source] → [Frame Processing] → [Video Encode] ┐
                                                                                                     ├→ [Mux] → [Container Stream]
[Audio Source] → [Audio Processing] → [Audio Encode] ┘


Playback 
────────────────────────────────────────────────────────

[Container Stream] → [Demux] → [Video Decode] → [Video Sink]
                                                      │
                                                      └→ [Audio Decode] → [Audio Sink] 
 

 

Sample image usage session:

jpg -yc a.jpg > a.yuv
1024x768
video/vsink -ki 1024x768 a.yuv
 
# usage: video/yuvconv -i fmt -s WxH [-o fmt] file
video/yuvconv -i yuv444 -s1024x768 a.yuv > b.yuv
video/vsink -ki 1024x768 b.yuv

 

Sample video usage session:

video/vsource -gn 1 > b.yuv
video/vsink b.yuv
video/vsource /dev/screen | video/vsink
 

 

Sink as a Screensaver / Image viewer 


Use composable programs to run screensaver or image viewer. DEL to exit the program.

Options:

g    grid ( 1x2 , 2x2 etc.) 
k    interactive (press any key to continue) 
l    loop
r    images per second (rate)
s    stretch 

 

Lessons Learnt

  1.  When a graphical program runs, it replaces the window label with the argument passed in initdraw(nil, nil, "label").
  2. riow has a CLI option -s label to set sticky window. Sticky windows are skipped while circling through open window.
  3. Plan 9 image is actually XBGR even though channel string is x8r8g8b8 and channel constant is XRGB32 on x86.
  4. Still images use YUV444 to maintain maximum details. Video uses YUV420 which sacrifices details for size. 


Code

 

 

 

Comments

Post a Comment

Popular posts from this blog

Plan 9 : The Infinity Notebook

Image
  Plan 9 is an Operating System (OS) from Bell Labs which is a successor to UNIX. It builds on the learnings from the previous Operating systems. Network computing and graphics is an integral part of the OS rather than an afterthought. It is modular and portable - it comes with a cross compiler . It is available, under MIT License , which anybody can use, learn and tweak. Features Everything is a file. Small Kernel:  Around 5MB size which can be built in under 2 minutes.  Singular Grid: All the computers running Plan 9 OS and connected together act like a singular grid. So there's no need for remote access solutions like VNC or RDP. This cool video shows a small glimpse of the possibilities. Process Isolation: Processes run within their own namespaces and are isolated from each other. So you can run rio (the window manager) within rio. Applications like Browsers don't need complicated sandboxing. And a crashing program is unlikely to bring down the OS. No DLL Hell : ...
Read more

Plan 9: Quick Boot with UEFI

Image
 UEFI ( Unified Extensible Firmware Interface) provides a quick way to set up a Plan 9 terminal  on modern hardware .  EFI System Partition (ESP) is a FAT32 partition. You should be able to modify it in most of the operating systems.     Run mk in /sys/src/boot/efi . aux/aout2efi converts an a.out file to an EFI executable.  For x86_64 hardware,  copy over the files bootx64.efi (EFI application) , 9pc64 (kernel) , plan9.ini  (configuration file) to \EFI\plan9\ directory in the ESP. If you want it be the default OS , you can create a symlink \EFI\boot to \EFI\plan9. Remember to include the full path to the kernel in plan9.ini. ( Note:  ESP is a FAT32 partition. Hence you must use backward slashes.) bootfile=\EFI\plan9\9pc64       When bootx64.efi fails to find the bootfile (e.g. kernel is missing, plan9.ini is missing or path is incorrect), you can provide the arguments at > prompt   followed by a boot command . Y...
Read more

Comics Builder in GNU Emacs

Image
  Generate comics strip from a script in GNU Emacs. First, create a folder containing artwork. You can use samples from [2]. It should have three subfolders - faces, poses and backgrounds. Images can be GIF, JPG or PNG. Filenames for face and poses follow a convention: <id>-<emotion>.<ext> . Please ensure you have atleast one file for neutral emotion . Set appropriate values for following variables: comics-artwork - Folder containing artwork comics-col-max - Maximum no. of columns (default: 2) comics-emotions - Alist of emotions and words used to describe them comics-avatars - A list of cons cells (FACE-ID . BODY-ID) Open a file containing the script (sample below) and run M-x comics. This will generate an SVG image for the comics strip which will fit the display window. Panels are rendered with fixed font-size so that it is always readable. Actors are assigned an avatar from comics-avatars sequentially as they appear in the script. Normally, all the actors are d...
Read more