ar-challenge | Augmented Reality Challenge for the Silicon Valley Computer | Computer Vision library

I've not done much with TargetDataLines yet, but I think I can offer a useful observation and suggestion.

First, the test you have written is probably measuring variance in the multi-threading algorithm, not slippage in the timing of the files. The way the JVM bounces back and forth between processing threads can be quite unpredictable. There is a good article on real time, low-latency coding in Java that you might read for background information.

Secondly, the way that Java uses blocking queues with audio IO provides a lot of stability. If it didn't, we'd hear all sorts of audio artifacts during playback or on our recordings.

Here is an idea to try: create a single runnable that has a while loop that processes an identical number of frames from both the TargetDataLine and the SourceDataLine in the same iteration. This runnable can be loosely coupled (use booleans to turn on/off the lines).

The main benefit is that you know that every loop iteration is producing coordinated data.

EDIT: Here are a couple examples of what I've done with frame counting: (1) I have an audio loop that counts frames as it processes. All timings are determined strictly by the number of frames processed. I never bother with taking readings from the position of the SDL. I've written a metronome, and it initiates a synthesized click every N frames (where N is based on tempo). At the Nth frame, the data for the synthesized click is mixed into the audio data that is being sent out of the SDL. The accuracy in timing that I have obtained by this method is outstanding.

Another application, on the Nth frame, I initiated a visual/graphical event. The graphics loop is usually set to 60fps and the audio to 44100 fps. The initiation is handled via loose coupling: a boolean for the event is flipped by the audio thread (nothing more than that, cluttering the audio thread with extraneous activity is hazardous, can lead to stuttering and dropouts). The graphics processing loop (aka "game loop") picks up the boolean change and handles it in its own time (60 fps). I've had some nice visual + aural synchronization occur this way, including having objects whose brightness tracks with the volume of the sound being played. This is similar to the digital VU meters that many have written using Java.

Depending on the level of accuracy you are hoping for, I think frame counting can be sufficient. I don't know of any other way, with Java, that provides as much accuracy.