RTSP live streaming on browsers using flv.js

I encounter the problem that a live RTSP would be better played on browsers. Here is my solution: joy4 [1] library is used to convert rtsp stream to flv stream and flv.js [2] library is used to play flv streams on modern browsers.

The benefits are clear. Not video decoding and encoding are performed and ffmpeg library is not depended on the server side. It costs almost no CPU usage when there’s only one client streaming. On the browser side, minimal CPU usage is also archieved considering mp4/h264 is well supported.

The browser side code looks like:

<script src="./flv.js"></script>
<video width="640" height="480" id="theVideo" controls></video>
<script>
    if (flvjs.isSupported()) {
        var videoElement = document.getElementById('theVideo');
        var flvPlayer = flvjs.createPlayer({
            type: 'flv',
            url: './video/'
        });
        flvPlayer.attachMediaElement(videoElement);
        flvPlayer.load();
        flvPlayer.play();
    }
</script>

The server side code looks like:

package main

import (
	"fmt"
	"io"
	"net/http"

	"github.com/nareix/joy4/av/avutil"
	"github.com/nareix/joy4/format"
	"github.com/nareix/joy4/format/flv"
	"github.com/nareix/joy4/format/rtsp"
)

type writeFlusher struct {
	httpflusher http.Flusher
	io.Writer
}

func (self writeFlusher) Flush() error {
	self.httpflusher.Flush()
	return nil
}

func init() {
	format.RegisterAll()
}

func main() {
	http.Handle("/", http.FileServer(http.Dir(".")))
	http.HandleFunc("/video/", func(w http.ResponseWriter, r *http.Request) {
		defer r.Body.Close()
		fmt.Println("serving:", r.RemoteAddr)
		if c, err := rtsp.Dial("rtsp://admin:admin@192.168.1.25:554/sub/av_stream"); nil != err {
			panic(err)
		} else {
			fmt.Println("rstp connected")
			c.Streams()
			defer c.Teardown()

			w.Header().Set("Content-Type", "video/flv")
			w.Header().Set("Transfer-Encoding", "chunked")
			w.Header().Set("Access-Control-Allow-Origin", "*")
			w.WriteHeader(200)

			fmt.Println("head wrote")
			flusher := w.(http.Flusher)
			flusher.Flush()

			muxer := flv.NewMuxerWriteFlusher(writeFlusher{httpflusher: flusher, Writer: w})

			if _, err := c.Streams(); nil != err {
				panic(err)
			}

			if err := avutil.CopyFile(muxer, c); nil != err {
				panic(err)
			}
		}

	})

	http.ListenAndServe(":8089", nil)
}

[1] https://github.com/nareix/joy4/
[2] https://github.com/Bilibili/flv.js/

Set up multiple SPI devices for Raspberry Pi 4B

According to the BCM2711 pin setups[1][2],Raspberry Pi 4B has 5 exported sets of SPI controllers: SPI3, SPI4, SPI0, SPI5 and SPI1. By default, SCLK of SPI4 and CE1_N of SPI1 share the same GPIO7 pin. When the Linux kernel is fired, a device tree and certain overlays are loaded. We can enable multiple SPI controllers like spi0-cs-overlay.dts in stock firmware[3] does.

We may disable the second chip enable line of SPI0 in first:

/dts-v1/;
/plugin/;
 
 
/ {
        compatible = "brcm,bcm2835";
 
        fragment@0 {
                target = <&amp;spi0_cs_pins>;
                frag0: __overlay__ {
                        brcm,pins = <8>;
                };
        };
 
        fragment@1 {
                target = <&amp;spi0>;
                frag1: __overlay__ {
                        cs-gpios = <&amp;gpio 8 1>;
                        status = "okay";
                };
        };
 
        __overrides__ {
                cs0_pin  = <&amp;frag0>,"brcm,pins:0",
                           <&amp;frag1>,"cs-gpios:4";
        };
};
# compile
dtc -I dts -O dtb -o spi0-1cs.dtbo spi0-1cs.dts
# copy to boot partition
sudo cp spi0-1cs.dtbo /boot/overlays

And then enable SPI3, SPI4, SPI5 and SPI1 which are already given in stock firmware. The following lines should be added into/boot/config.txt. Attention must be paid that we should comment out the default spi=on:

#dtparam=spi=on
dtoverlay=spi0-1cs
dtoverlay=spi3-1cs
dtoverlay=spi4-1cs
dtoverlay=spi1-1cs

We can also apply the previous trick to other device tree overlays. Here I added additional three MCP2515 CAN controllers[4]:

dtoverlay=mcp2515-can0,oscillator=8000000,interrupt=24
dtoverlay=mcp2515-can3,oscillator=8000000,interrupt=25
dtoverlay=mcp2515-can4,oscillator=8000000,interrupt=26
dtoverlay=mcp2515-can1,oscillator=8000000,interrupt=27

Log from dmesg indicated that four CAN controllers were all working smoothly:

pi@raspberrypi:~$ dmesg | grep spi
[    4.387592] spi-bcm2835 fe204600.spi: no tx-dma configuration found - not using dma mode
[    4.391881] spi-bcm2835 fe204600.spi: chipselect 0 already in use
[    4.391898] spi_master spi3: spi_device register error /soc/spi@7e204600/spidev@0
[    4.391915] spi_master spi3: Failed to create SPI device for /soc/spi@7e204600/spidev@0
[    4.392332] spi-bcm2835 fe204800.spi: no tx-dma configuration found - not using dma mode
[    4.394967] spi-bcm2835 fe204800.spi: chipselect 0 already in use
[    4.394982] spi_master spi4: spi_device register error /soc/spi@7e204800/spidev@0
[    4.394999] spi_master spi4: Failed to create SPI device for /soc/spi@7e204800/spidev@0
[    4.419969] spi-bcm2835aux fe215080.spi: chipselect 0 already in use
[    4.419986] spi_master spi1: spi_device register error /soc/spi@7e215080/spidev@0
[    4.420005] spi_master spi1: Failed to create SPI device for /soc/spi@7e215080/spidev@0
[    5.351820] mcp251x spi0.0 can0: MCP2515 successfully initialized.
[    5.368085] mcp251x spi3.0 can1: MCP2515 successfully initialized.
[    5.379651] mcp251x spi4.0 can2: MCP2515 successfully initialized.
[    5.390782] mcp251x spi1.0 can3: MCP2515 successfully initialized.

[1] https://elinux.org/RPi_BCM2711_GPIOs
[2] https://www.raspberrypi.org/documentation/usage/gpio/
[3] https://github.com/raspberrypi/linux/blob/rpi-5.4.y/arch/arm/boot/dts/overlays/spi0-cs-overlay.dts
[4] https://github.com/raspberrypi/linux/blob/rpi-5.4.y/arch/arm/boot/dts/overlays/mcp2515-can0-overlay.dts

OpenWrt stable 19.07.2 firmware for Xiaomi Mi Router 4A Gigabit version

I followed instructions shared in [1] and successfully let the device operate. I deployed snapshot build at beginning, but sadly I ran into a bug[2]. Obviously, stable version is the better choice. I put the mir3g-v2 description file back to the stable branch[3] and built with luci web interface. The built firmware could be downloaded here[4].

Update: There’s an open-source tool [5] to get the root shell from the device without hijacking the chips. Please leave a comment if you need video guideline.

CH341A on Mi Router 4A Gigabit version

[1] https://forum.openwrt.org/t/xiaomi-mi-router-4a-gigabit-edition-r4ag-r4a-gigabit-fully-supported-but-requires-overwriting-spi-flash-with-programmer/36685
[2] https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=fdfca33350150644481096f1c7a80db2b670cdec
[3] https://sharing-1252192454.cos.ap-shanghai.myqcloud.com/openwrt/mir4a-gigabyte/backport%20to%2019.07.2.diff
[4] https://sharing-1252192454.cos.ap-shanghai.myqcloud.com/openwrt/mir4a-gigabyte/openwrt-ramips-mt7621-xiaomi_mir3g-v2-squashfs-factory.bin
[5] https://github.com/acecilia/OpenWRTInvasion

Send template message in WeChat mini program

According to the WeChat Development document, a field form_id should be presented when requesting a template message to be sent to the user. But the problem is that one template message for one corresponding form can’t meet all the needs. So we need to keep collecting available form_ids for future template message sending.

Continue reading Send template message in WeChat mini program