A few days a go I was tweaking ScreenStudio 2.0. The main idea is to create a dynamic overlay a bit like WebcamStudio without having to use a lot of CPU resources.
WebcamStudio is great but do required a powerful computer to make it work. On the other hand, ScreenStudio is more limited but do have the advantage of being simple to broadcast live your desktop over Twitch and Youtube.
ScreenStudio do support a static overlay created by rendering an HTML file using a JLabel (a label component in Java Swing). The result is pretty good but do lack dynamic content. I've tried several ways without major success. That is until a few days ago.
I knew that FFMpeg is able to read from many sources. I had, in the past, used a TCP/IP connections to get a BufferedImage to FFMpeg. The process is slow and do required a lot of code to handle the connections.
I also tried using the OutputStream for the Process to output the content to FFMpeg but it does not work reliably.
Then it occured to me that I could use a fifo file using the command "mkfifo" in linux to output a BufferedImage directly to FFMpeg.
The command is quite simple:
ffmpeg -re -r 10 -s 400x480 -f rawvideo -pix_fmt bgr24 -i /tmp/tempfile.bgr24 test.flv
I just needed to create a temporary fifo file, write to it in a format that FFMpeg could understand. Initially, I used ImageIO.write to dump the BufferedImage into a JPG format. It was working well be do required a bit more processing power due to the JPEG encoding for each frame...
The fasted way do to it was to output a raw file in RGB format, or preferably in BGR format since FFMpeg do support this particular data.
The end result is quite nice and simple:
1 - Create a temp fifo file to write into
2 - Create a BufferedImage that will be used to paint a JPanel
3 - Dump the byte array from the BufferedImage into the fifo file
4 - Let FFMpeg read and encode the data into a video
5 - While the process is running, update the content of the JPanel with what ever you want...
Here's the full source code that you can test for your self, in Java 8:
/*
* Use FFMPEG to procude a video file:
* ffmpeg -re -r 10 -s 400x480 -f rawvideo -pix_fmt bgr24 -i /tmp/tempfile.bgr24 test.flv
*/
package screenstudio.sources;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JPanel;
import javax.swing.JTextField;
/**
*
* @author patrick
*/
public class OverlayPipe implements Runnable {
// Make a file pipe...
private File mPipe = null;
private JPanel mPanel = null;
private Thread mThread = null;
private long mFPS = 10;
private boolean stopMe = false;
private boolean mIsRunning = false;
public OverlayPipe(JPanel panel, long fps) {
mPanel = panel;
mFPS = fps;
}
public void start() {
mThread = new Thread(this);
mThread.start();
}
@Override
public void run() {
mIsRunning = true;
stopMe = false;
java.io.OutputStream output = null;
try {
mPipe = createPipeFile();
// Pipe created. so we need to paint
// the panel in the fifo each x ms seconds..
long nextTimeStamp;
long delay;
// Use a BGR 24 bits images as ffmpeg will read -pix_format BGR24
BufferedImage img = new BufferedImage(mPanel.getWidth(),mPanel.getHeight(),BufferedImage.TYPE_3BYTE_BGR);
Graphics2D graphics = img.createGraphics();
output = new FileOutputStream(mPipe);
while (!stopMe){
nextTimeStamp = System.currentTimeMillis() + (1000/mFPS);
mPanel.paint(graphics);
byte[] imageBytes = ((DataBufferByte) img.getRaster().getDataBuffer()).getData();
output.write(imageBytes);
output.flush();
//Try to sleep just what is needed to keep a constant fps
delay = nextTimeStamp - System.currentTimeMillis();
if (delay >0){
Thread.sleep(delay);
}
}
graphics.dispose();
output.close();
mPipe.delete();
mPipe = null;
} catch (IOException | InterruptedException ex) {
Logger.getLogger(OverlayPipe.class.getName()).log(Level.SEVERE, null, ex);
if (output != null){
try {
output.close();
} catch (IOException ex1) {
Logger.getLogger(OverlayPipe.class.getName()).log(Level.SEVERE, null, ex1);
}
}
if (mPipe != null){
if (mPipe.exists()){
mPipe.delete();
}
mPipe = null;
}
}
mIsRunning = false;
}
public void stop() {
stopMe =true;
}
public boolean isRunning(){
return mIsRunning;
}
// Create a temp fifo file for output
private File createPipeFile() throws IOException, InterruptedException {
File tempFile = File.createTempFile("screenstudio", ".bgr24");
tempFile.deleteOnExit();
java.lang.Process p = Runtime.getRuntime().exec("/usr/bin/mkfifo", new String[]{tempFile.getAbsolutePath()});
p.waitFor();
p.destroy();
System.out.println("Output Overlay: " + tempFile.getAbsolutePath());
return tempFile;
}
//For testing...
public static void main(String[] args) {
JPanel p = new JPanel();
p.setSize(400, 480);
JTextField text = new JTextField();
text.setSize(400,30);
p.add(text);
OverlayPipe op = new OverlayPipe(p,10);
op.start();
try {
Thread.sleep(500);
} catch (InterruptedException ex) {
Logger.getLogger(OverlayPipe.class.getName()).log(Level.SEVERE, null, ex);
}
for(int i = 0;i<10000 && op.isRunning();i++){
text.setText(new java.util.Date().toString());
try {
Thread.sleep(1000);
} catch (InterruptedException ex) {
Logger.getLogger(OverlayPipe.class.getName()).log(Level.SEVERE, null, ex);
}
}
op.stop();
}
}
No comments:
Post a Comment