An image can be read from disk using read-image which uses the ImageMagick library to load the image.
fubk.png
(use-modules (aiscm magick) (aiscm xorg))
(show (read-image "fubk.png"))
Analogous write-image writes an image to disk.
(use-modules (aiscm sequence) (aiscm magick) (aiscm xorg))
(write-image (arr (32 255 96 255) (255 160 255 224)) "pattern.png")
scaled-pattern.png
As shown above, you can display images using the method show as follows:
(use-modules (aiscm magick) (aiscm xorg))
(show (read-image "fubk.png"))
One can also display a list of images:
fubk-colours.png
(use-modules (aiscm magick) (aiscm xorg) (aiscm rgb) (aiscm jit))
(define img (read-image "fubk.png"))
(show (list (* (rgb 1 0 0) img) (* (rgb 0 1 0) img) (* (rgb 0 0 1) img)))
It is also possible to display a video using the show method:
(use-modules (oop goops) (aiscm v4l2) (aiscm xorg))
(define v (make <v4l2>))
(show (lambda _ (read-image v)))
(destroy v)
A function returning lists of images can be used to display multiple videos synchronously.
(use-modules (oop goops) (aiscm v4l2) (aiscm sequence) (aiscm rgb) (aiscm xorg))
(define v (make <v4l2>))
(show (lambda _ (let [(img (to-array (read-image v)))] (list (red img) (green img) (blue img)))))
(destroy v)
If necessary, one can also handle the display and window objects directly. Possible types of output are IO-XIMAGE, IO-OPENGL, and IO-XVIDEO.
(use-modules (oop goops) (aiscm v4l2) (aiscm xorg))
(define v (make <v4l2>))
(define d (make <xdisplay> #:name ":0.0"))
(define w (make <xwindow> #:display d #:shape '(640 480) #:io IO-XVIDEO))
(title= w "Test")
(show w)
(while (not (quit? d)) (write-image (read-image v) w) (process-events d))
(quit= d #f)
(hide w)
(destroy d)
(destroy v)
v4l2.jpg
As shown above already, you can open a camera and grab a frame as follows.
(use-modules (oop goops) (aiscm v4l2))
(define v (make <v4l2>))
(read-image v)
; #<<image> YUY2 (640 480)>
(destroy v)
It is also possible to specify the device, a channel, and a closure for selecting the video mode.
(use-modules (oop goops) (ice-9 rdelim) (aiscm v4l2))
(define (select formats)
(for-each (lambda (i mode) (format #t "~a: ~a~&" i mode))
(iota (length formats))
formats)
(format #t "> ") (list-ref formats (string->number (read-line (current-input-port)))))
(define v (make <v4l2> #:device "/dev/video0" #:channel 0 #:select select))
(read-image v)
(destroy v)
The following example program creates a sine wave and outputs it to the audio device.
(use-modules (oop goops) (aiscm pulse) (aiscm int) (aiscm sequence))
(define samples (to-array <sint> (map (lambda (t) (round (* (sin (/ (* t 1000 2 3.1415926) 44100)) 20000))) (iota 441))))
(define output (make <pulse-play> #:typecode <sint> #:channels 1 #:rate 44100))
(channels output)
;1
(rate output)
;44100
(for-each (lambda _ (write-audio samples output)) (iota 300))
(drain output)
The drain method waits for the content of the audio buffer to finish playing. The method flush (not shown here) can be used to empty the audio buffer.
Audio data can be recorded in a similar fashion. The following example records 3 seconds of audio data and then plays it back.
(use-modules (oop goops) (aiscm int) (aiscm pulse))
(define record (make <pulse-record> #:typecode <sint> #:channels 2 #:rate 44100))
(define play (make <pulse-play> #:typecode <sint> #:channels 2 #:rate 44100))
(for-each (lambda _ (write-audio (read-audio record 4410) play)) (iota 30))
(drain play)
The following example shows how to use the FFmpeg interface to open and view a video. The video presentation time stamps are used to display the video at the correct speed. The method latency is used to determine the delay of the audio buffer.
(use-modules (aiscm ffmpeg) (aiscm xorg) (aiscm util))
; Creative commons audio-video sync test video https://www.youtube.com/watch?v=GKBKa9Za-FQ
(define video (open-ffmpeg-input "av-sync.mp4"))
(define time (clock))
(show
(lambda (dsp)
(format #t "video pts = ~8,2f, clock = ~8,2f~&" (video-pts video) (elapsed time))
(synchronise (read-image video) (- (video-pts video) (elapsed time)) (event-loop dsp))))
The method pts= can be used to seek to an absolute position in audio/video streams:
(use-modules (aiscm ffmpeg) (aiscm xorg))
; Creative commons audio-video sync test video https://www.youtube.com/watch?v=GKBKa9Za-FQ
(define video (open-ffmpeg-input "av-sync.mp4"))
(pts= video 15)
(show (read-image video))
Note that FFmpeg also supports network streaming of video data. I.e. the following example will play the Sintel short film from a web server.
(use-modules (oop goops) (aiscm ffmpeg) (aiscm xorg) (aiscm pulse) (aiscm image) (aiscm util))
(define video (open-ffmpeg-input "http://peach.themazzone.com/durian/movies/sintel-1024-surround.mp4"))
(define pulse (make <pulse-play> #:rate (rate video) #:channels (channels video) #:typecode (typecode video) #:latency 0.1))
(show
(lambda (dsp)
(while (< (audio-pts video) (+ (video-pts video) 0.2)) (write-audio (or (read-audio video) (break)) pulse))
(synchronise (read-image video) (- (video-pts video) (- (audio-pts video) (latency pulse))) (event-loop dsp))))
(drain pulse)
One can play samples from an audio file by passing them to the audio device using the write-audio method. It is also possible to pass a function returning consecutive audio samples as shown below.
(use-modules (oop goops) (aiscm ffmpeg) (aiscm pulse))
(define audio (open-ffmpeg-input "test.mp3"))
(define output (make <pulse-play> #:rate (rate audio) #:channels (channels audio) #:typecode (typecode audio)))
(write-audio (lambda _ (read-audio audio)) output)
(drain output)