Plan 9: Real Time Streaming

Stream a single window or the whole screen via Real Time Streaming Protocol (RTSP).

This uses MJPEG stream over TCP-interleave. Plan 9 /dev/screen and /dev/window produces RGBA byte stream with Plan 9 headers. This is converted to JPEG using tojpg and streamed using RTSP.

In the video, host port 12554 port is being forwarded to 554 in the Plan 9 VM. 

ffplay -rtsp_transport tcp rtsp://localhost:12554/stream1 
 

 

Code

 

Youtube Live

Use Real-Time Messaging Protocol (RTMP) to feed into Youtube / Twitch Live.

video/hj264 -f 25 /dev/screen | video/rtmp -a /dev/audio rtmp://.... rtmp://...     
 

Note: 

  1. The encoder and the publisher programs work with the Inter Video Format (IVF) container.
  2. If you are looking for a simple H264 encoder to port to another platform, minih264e is a good place to start.


 Demo Live URL: https://youtube.com/live/dvoWNjdwMCo


Lessons Learnt

  • MJPEG and H264 are the most widely supported formats. 
  • YUV422 (type 0 or 64) and YUV420 (type 1 or 65) with q=50 are the most widely supported formats for MJPEG streaming. Don't use YUV444.
  • q factor is used to calculate quantization table DQT. You must use the formula defined in the RFC since the table itself is NOT transmitted.
  • RTSP client displays a frame only AFTER the arrival of next frame and NOT after receiving the end of frame marker (eg. FF D9 for MJPEG). Be mindful when you are testing a single frame. You need to close the connection to view it.
  • Human eyes are more sensitive to brightness than to color - this is the basis of Chroma-subsampling.
  • We use XRGB instead of simply RGB for 32 bit alignment. X might be alpha or be unused.
  • YUV is called rawvideo format in ffmpeg and the sub-sampling is specified as the pixel_format.
  • XRGB to YUV420 compression ratio is 4:1.5 .
  • Youtube Live NEEDS BOTH audio and video streams.
  • Use readn() to deal with fixed size data e.g. a frame.
 

References

  1. RTP Payload Format for JPEG-compressed Video (RFC2435) 
  2. Anatomy of a JPEG 
  3. JPEG Syntax and structure
  4. Chroma Subsampling
  5. FFMPEG Implementation
  6. GStreamer Implementation    

 

Comments

Post a Comment

Popular posts from this blog

Plan 9 : The Infinity Notebook

Image
  Plan 9 is an Operating System (OS) from Bell Labs which is a successor to UNIX. It builds on the learnings from the previous Operating systems. Network computing and graphics is an integral part of the OS rather than an afterthought. It is modular and portable - it comes with a cross compiler . It is available, under MIT License , which anybody can use, learn and tweak. Features Everything is a file. Small Kernel:  Around 5MB size which can be built in under 2 minutes.  Singular Grid: All the computers running Plan 9 OS and connected together act like a singular grid. So there's no need for remote access solutions like VNC or RDP. This cool video shows a small glimpse of the possibilities. Process Isolation: Processes run within their own namespaces and are isolated from each other. So you can run rio (the window manager) within rio. Applications like Browsers don't need complicated sandboxing. And a crashing program is unlikely to bring down the OS. No DLL Hell : ...
Read more

Emacs: Binary File Viewer

Image
  Quick visualization tool for Binary data using C header definition. Semantic parses the C header file and generates a type definition. The type definition is passed to bindat-unpack along with the binary data. The result is displayed using speedbar . ;; Note: The command works on the binary data in the current buffer.   M-x hexl-form   (hexl-form)      The image shows the example included in the commentary section of bindat.el .   Conversions (format "%02X" 255) ;; => "FF" (string-to-number "FF" 16) ;; => 255   ;; Little endian unpacking (let* ((bindat-raw [1 2 3 4 5 6 7 8])           (bindat-idx 0))   (bindat--unpack-u64r))   ;; Little endian packing (let* ((bindat-raw [0 0 0 0 0 0 0 0])           (bindat-idx 0))   (bindat--pack-u64r #x0102030405060708)   bindat-raw)      Insert Binary Data Emacs creates a multibyte buffer ...
Read more

Plan 9 Remote File Access from Emacs

Image
  Plan 9 Operating System uses 9p protocol for file access. This is an Elisp implementation of the protocol. Plan 9 ( 9front distribution) is running in QEMU with NAT networking. Local port 12564 is forwarded to 9fs service port 564 in the virtual machine.   Code https://gitlab.com/atamariya/emacs/-/blob/dev/lisp/net/plan9.el   Troubleshooting Plan 9 connection Ensure you are booting with  -a tcp!*!564  parameters. (Tip: You can add these to your /n/9fat/plan9.ini) Ensure you have configured the network interface   ip/ipconfig   Ensure you have an IP address  cat /net/ndb  Ensure you are running cpu+auth server  cpurc  (Optional) Start graphics mode  screenrc   (Optional) Start window manager  rio   (Optional) List open connections  netstat    (Optional) Monitor network traffic  snoopy   (Optional) Debug authentication  auth/debug  (Optional)...
Read more