Linux + webcam: Poor man’s DIY surveillance camera
Introduction
Due to an incident that is beyond the scope of this blog, I wanted to put a 24/7 camera that watched a certain something, just in case that incident repeated itself.
Having a laptop that I barely use, and a cheap e-bay web camera, I thought I set up something and let ffmpeg do the job.
I’m not sure if a Raspberry Pi would be up for this job, even when connected to an external hard disk through USB. It depends much on how well ffmpeg performs on that platform. Haven’t tried. The laptop’s clear advantage is when there’s a brief power outage.
Overall verdict: It’s as good as the stability of the USB connection with the camera.
Note to self: I keep this in the misc/utils git repo, under surveillance-cam/.
Warming up
Show the webcam’s image on screen, the ffmpeg way:
$ ffplay -f video4linux2 /dev/video0
Let ffmpeg list the formats:
$ ffplay -f video4linux2 -list_formats all /dev/video0
Or with a dedicated tool:
# apt install v4l-utils
and then
$ v4l2-ctl --list-formats-ext -d /dev/video0
Possibly also use “lsusb -v” on the device: It lists the format information, not necessarily in a user-friendly way, but that’s the actual source of information.
Get all parameters that can be tweaked:
$ v4l2-ctl --all
See an example output for this command at the bottom of this post.
If control over the exposure time is available, it will be listed as “exposure_absolute” (none of the webcams I tried had this). The exposure time is given in units of 100µs (see e.g. the definition of V4L2_CID_EXPOSURE_ABSOLUTE).
Get a specific parameter, e.g. brightness
$ v4l2-ctl --get-ctrl=brightness brightness: 137
Set the control (can be done while the camera is capturing video)
$ v4l2-ctl --set-ctrl=brightness=255
Continuous capturing
This is a simple bash script that creates .mp4 files from the captured video:
#!/bin/bash
OUTDIR=/extra/videos SRC=/dev/v4l/by-id/usb-Generic*
DURATION=3600 # In seconds
while [ 1 ]; do
TIME=`date +%F-%H%M%S`
if ! ffmpeg -f video4linux2 -i $SRC -t $DURATION -r 10 $OUTDIR/video-$TIME.mp4 < /dev/null ; then
echo 2-2 | sudo tee /sys/bus/usb/drivers/usb/unbind
echo 2-2 | sudo tee /sys/bus/usb/drivers/usb/bind
sleep 5;
fi
done
Comments on the script:
- To make this a real surveillance application, there must be another script that deletes old files, so that the disk isn’t full. My script on this matter is so hacky, that I left it out here.
- The real problem I encountered was occasional USB errors. They happened every now and then, without any specific pattern. Sometimes the camera disconnected briefly and reconnected right away, sometimes it failed to come back for a few minutes. Once in a week or so, it didn’t come back at all, and only a lot of USB errors appeared in the kernel log, so a reboot was required. This is most likely some kind of combination of cheap hardware, a long and not so good USB cable and maybe hardware + kernel driver issues. I don’t know. This wasn’t important enough to solve in a bulletproof way.
- Because of these USB errors, those two “echo 2-2″ commands attempt to reset the USB port if ffmpeg fails, and then sleep 5 seconds. The “2-2″ is the physical position of the USB port to which the USB camera was connected. Ugly hardcoding, yes. I know for sure that these commands were called occasionally, but whether this helped, I’m not sure.
- Also because of these disconnections, the length of the videos wasn’t always 60 minutes as requested. But this doesn’t matter all that much, as long as the time between the clips is short. Which it usually was (less than 5 seconds, the result of a brief disconnection).
- Note that the device file for the camera is found using a /dev/v4l/by-id/ path rather than /dev/video0, not just to avoid mixing between the external and built-in webcam: There were sporadic USB disconnections after which the external webcam ended up as /dev/video2. And then back to /dev/video1 after the next disconnection. The by-id path remained constant in the sense that it could be found with the * wildcard.
- Frame rate is always a dilemma, as it ends up influencing the file’s size, and hence how long back videos are stored. At 5 fps, an hour long .mp4 took about 800 MB for daytime footage, and much less than so during night. At 10 fps, it got up to 1.1 GB, so by all means, 10 fps is better.
- Run the recording on a text console, rather than inside a terminal window inside X-Windows (i.e. use Ctrl-Alt-F1 and Ctrl-Alt-F7 to go back to X). This is because the graphical desktop crashed at some point — see below on why. So if this happens again, the recording will keep going.
- For the purpose of running ffmpeg without a console (i.e. run in the background with an “&” and then log out), note that the ffmpeg command has a “< /dev/null”. Otherwise ffmpeg expects to be interactive, meaning it does nothing if it runs in the background. There’s supposed to be a -nostdin flag for this, and ffmpeg recognized it on my machine, but expected a console nevertheless. So I went for the old method.
How a wobbling USB camera crashes X-Windows
First, the spoiler: I solved this problem by putting a physical weight on the USB cable, close to the plug. This held the connector steady in place, and the vast majority of the problems were gone.
I also have a separate post about how I tried to make Linux ignore the offending bogus keyboard from being. Needless to say, that failed (because either you ban the entire USB device or you don’t ban at all).
This is the smoking gun in /var/log/Xorg.0.log: Lots of
[1194182.076] (II) config/udev: Adding input device USB2.0 PC CAMERA: USB2.0 PC CAM (/dev/input/event421) [1194182.076] (**) USB2.0 PC CAMERA: USB2.0 PC CAM: Applying InputClass "evdev keyboard catchall" [1194182.076] (II) Using input driver 'evdev' for 'USB2.0 PC CAMERA: USB2.0 PC CAM' [1194182.076] (**) USB2.0 PC CAMERA: USB2.0 PC CAM: always reports core events [1194182.076] (**) evdev: USB2.0 PC CAMERA: USB2.0 PC CAM: Device: "/dev/input/event421" [1194182.076] (--) evdev: USB2.0 PC CAMERA: USB2.0 PC CAM: Vendor 0x1908 Product 0x2311 [1194182.076] (--) evdev: USB2.0 PC CAMERA: USB2.0 PC CAM: Found keys [1194182.076] (II) evdev: USB2.0 PC CAMERA: USB2.0 PC CAM: Configuring as keyboard [1194182.076] (EE) Too many input devices. Ignoring USB2.0 PC CAMERA: USB2.0 PC CAM [1194182.076] (II) UnloadModule: "evdev"
and at some point the sad end:
[1194192.408] (II) config/udev: Adding input device USB2.0 PC CAMERA: USB2.0 PC CAM (/dev/input/event423) [1194192.408] (**) USB2.0 PC CAMERA: USB2.0 PC CAM: Applying InputClass "evdev keyboard catchall" [1194192.408] (II) Using input driver 'evdev' for 'USB2.0 PC CAMERA: USB2.0 PC CAM' [1194192.408] (**) USB2.0 PC CAMERA: USB2.0 PC CAM: always reports core events [1194192.408] (**) evdev: USB2.0 PC CAMERA: USB2.0 PC CAM: Device: "/dev/input/event423" [1194192.445] (EE) [1194192.445] (EE) Backtrace: [1194192.445] (EE) 0: /usr/bin/X (xorg_backtrace+0x48) [0x564128416d28] [1194192.445] (EE) 1: /usr/bin/X (0x56412826e000+0x1aca19) [0x56412841aa19] [1194192.445] (EE) 2: /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f6e4d8b4000+0x10340) [0x7f6e4d8c4340] [1194192.445] (EE) 3: /usr/lib/xorg/modules/input/evdev_drv.so (0x7f6e45c4c000+0x39f5) [0x7f6e45c4f9f5] [1194192.445] (EE) 4: /usr/lib/xorg/modules/input/evdev_drv.so (0x7f6e45c4c000+0x68df) [0x7f6e45c528df] [1194192.445] (EE) 5: /usr/bin/X (0x56412826e000+0xa1721) [0x56412830f721] [1194192.446] (EE) 6: /usr/bin/X (0x56412826e000+0xb731b) [0x56412832531b] [1194192.446] (EE) 7: /usr/bin/X (0x56412826e000+0xb7658) [0x564128325658] [1194192.446] (EE) 8: /usr/bin/X (WakeupHandler+0x6d) [0x5641282c839d] [1194192.446] (EE) 9: /usr/bin/X (WaitForSomething+0x1bf) [0x5641284142df] [1194192.446] (EE) 10: /usr/bin/X (0x56412826e000+0x55771) [0x5641282c3771] [1194192.446] (EE) 11: /usr/bin/X (0x56412826e000+0x598aa) [0x5641282c78aa] [1194192.446] (EE) 12: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xf5) [0x7f6e4c2f3ec5] [1194192.446] (EE) 13: /usr/bin/X (0x56412826e000+0x44dde) [0x5641282b2dde] [1194192.446] (EE) [1194192.446] (EE) Segmentation fault at address 0x10200000adb [1194192.446] (EE) Fatal server error: [1194192.446] (EE) Caught signal 11 (Segmentation fault). Server aborting [1194192.446] (EE)
The thing is that webcam presents itself as a keyboard, among others. I guess the chipset has inputs for control buttons (which the specific webcam doesn’t have), so as the USB device goes on and off, X windows registers the nonexistent keyboard on and off, and eventually some bug causes it to crash (note that number of the event device is 423, so there were quite a few on and offs). It might very well be that the camera camera connected, started some kind of connection event handler, which didn’t finish its job before it disconnected. Somewhere in the code, the handler fetched information that didn’t exist, it got a bad pointer instead (NULL?) and used it. Boom. Just a wild guess, but this is the typical scenario.
The crash can be avoided by making X windows ignore this “keyboard”. I did this by adding a new file named /usr/share/X11/xorg.conf.d/10-nocamera.conf as follows:
# Ignore bogus button on webcam Section "InputClass" Identifier "Blacklist USB webcam button as keyboard" MatchUSBID "1908:2311" Option "Ignore" "on" EndSection
This way, X windows didn’t fiddle with the bogus buttons, and hence didn’t care if they suddenly went away.
Anyhow, it’s a really old OS (Ubuntu 14.04.1) so this bug might have been solved long ago.
Accumulation of /dev/input/event files
Another problem with this wobbling is that /dev/input/ becomes crowded with a lot of eventN files:
$ ls /dev/input/event* /dev/input/event0 /dev/input/event267 /dev/input/event295 /dev/input/event1 /dev/input/event268 /dev/input/event296 /dev/input/event10 /dev/input/event269 /dev/input/event297 /dev/input/event11 /dev/input/event27 /dev/input/event298 /dev/input/event12 /dev/input/event270 /dev/input/event299 /dev/input/event13 /dev/input/event271 /dev/input/event3 /dev/input/event14 /dev/input/event272 /dev/input/event30 /dev/input/event15 /dev/input/event273 /dev/input/event300 /dev/input/event16 /dev/input/event274 /dev/input/event301 /dev/input/event17 /dev/input/event275 /dev/input/event302 /dev/input/event18 /dev/input/event276 /dev/input/event303 /dev/input/event19 /dev/input/event277 /dev/input/event304 /dev/input/event2 /dev/input/event278 /dev/input/event305 /dev/input/event20 /dev/input/event279 /dev/input/event306 /dev/input/event21 /dev/input/event28 /dev/input/event307 /dev/input/event22 /dev/input/event280 /dev/input/event308 /dev/input/event23 /dev/input/event281 /dev/input/event309 /dev/input/event24 /dev/input/event282 /dev/input/event31 /dev/input/event25 /dev/input/event283 /dev/input/event310 /dev/input/event256 /dev/input/event284 /dev/input/event311 /dev/input/event257 /dev/input/event285 /dev/input/event312 /dev/input/event258 /dev/input/event286 /dev/input/event313 /dev/input/event259 /dev/input/event287 /dev/input/event314 /dev/input/event26 /dev/input/event288 /dev/input/event315 /dev/input/event260 /dev/input/event289 /dev/input/event316 /dev/input/event261 /dev/input/event29 /dev/input/event4 /dev/input/event262 /dev/input/event290 /dev/input/event5 /dev/input/event263 /dev/input/event291 /dev/input/event6 /dev/input/event264 /dev/input/event292 /dev/input/event7 /dev/input/event265 /dev/input/event293 /dev/input/event8 /dev/input/event266 /dev/input/event294 /dev/input/event9
Cute, huh? And this is even before there was a problem. So what does X windows make of this?
$ xinput list
⎡ Virtual core pointer id=2 [master pointer (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ ELAN Touchscreen id=9 [slave pointer (2)]
⎜ ↳ SynPS/2 Synaptics TouchPad id=13 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ Power Button id=6 [slave keyboard (3)]
↳ Video Bus id=7 [slave keyboard (3)]
↳ Power Button id=8 [slave keyboard (3)]
↳ Lenovo EasyCamera: Lenovo EasyC id=10 [slave keyboard (3)]
↳ Ideapad extra buttons id=11 [slave keyboard (3)]
↳ AT Translated Set 2 keyboard id=12 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=14 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=15 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=16 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=17 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=18 [slave keyboard (3)]
↳ USB 2.0 PC Cam id=19 [slave keyboard (3)]
Now, let me assure you that there were not six webcams connected when I did this. Actually, not a single one.
Anyhow, I didn’t dig further into this. The real problem is that all of these /dev/input/event files have the same major. Which means that when there are really a lot of them, the system runs out of minors. So if the normal kernel log for plugging in the webcam was this,
usb 2-2: new high-speed USB device number 22 using xhci_hcd usb 2-2: New USB device found, idVendor=1908, idProduct=2311 usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0 usb 2-2: Product: USB2.0 PC CAMERA usb 2-2: Manufacturer: Generic uvcvideo: Found UVC 1.00 device USB2.0 PC CAMERA (1908:2311) uvcvideo 2-2:1.0: Entity type for entity Processing 2 was not initialized! uvcvideo 2-2:1.0: Entity type for entity Camera 1 was not initialized! input: USB2.0 PC CAMERA: USB2.0 PC CAM as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/input/input274
after all minors ran out, I got this:
usb 2-2: new high-speed USB device number 24 using xhci_hcd
usb 2-2: New USB device found, idVendor=1908, idProduct=2311
usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 2-2: Product: USB2.0 PC CAMERA
usb 2-2: Manufacturer: Generic
uvcvideo: Found UVC 1.00 device USB2.0 PC CAMERA (1908:2311)
uvcvideo 2-2:1.0: Entity type for entity Processing 2 was not initialized!
uvcvideo 2-2:1.0: Entity type for entity Camera 1 was not initialized!
media: could not get a free minor
And then immediately after:
systemd-udevd[4487]: Failed to apply ACL on /dev/video2: No such file or directory systemd-udevd[4487]: Failed to apply ACL on /dev/video2: No such file or directory
Why these eventN files aren’t removed is unclear. The kernel is pretty old, v4.14, so maybe this has been fixed since.
Sample output of v412-all
This is small & junky webcam. Clearly no control over exposure time.
$ v4l2-ctl --all -d /dev/v4l/by-id/usb-Generic_USB2.0_PC_CAMERA-video-index0
Driver Info (not using libv4l2):
Driver name : uvcvideo
Card type : USB2.0 PC CAMERA: USB2.0 PC CAM
Bus info : usb-0000:00:14.0-2
Driver version: 4.14.0
Capabilities : 0x84200001
Video Capture
Streaming
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
Width/Height : 640/480
Pixel Format : 'YUYV'
Field : None
Bytes per Line: 1280
Size Image : 614400
Colorspace : Unknown (00000000)
Custom Info : feedcafe
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 640, Height 480
Default : Left 0, Top 0, Width 640, Height 480
Pixel Aspect: 1/1
Selection: crop_default, Left 0, Top 0, Width 640, Height 480
Selection: crop_bounds, Left 0, Top 0, Width 640, Height 480
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0
brightness (int) : min=0 max=255 step=1 default=128 value=128
contrast (int) : min=0 max=255 step=1 default=130 value=130
saturation (int) : min=0 max=255 step=1 default=64 value=64
hue (int) : min=-127 max=127 step=1 default=0 value=0
gamma (int) : min=1 max=8 step=1 default=4 value=4
power_line_frequency (menu) : min=0 max=2 default=1 value=1
sharpness (int) : min=0 max=15 step=1 default=13 value=13
backlight_compensation (int) : min=1 max=5 step=1 default=1 value=1