首页 新闻 赞助 找找看

JavaCV 解码 H264流到图片

0
悬赏园豆:50 [已关闭问题] 关闭于 2019-12-24 15:11

需求

客户端接收来自服务器的H264视频流,然后要实时的显示到客户端页面,比如JavaFX。

问题

我可以正确的接收到H264字节流,也可以通过JavaCV进行解码到图片,但是这个过程是阻塞的,(当我断开Socket时才会进行渲染)。

客户端代码

public class FFmpegFXImageDecoder {
    private FFmpegFXImageDecoder() {
    }

    public static void streamToImageView(ImageView view, int port, int socketBacklog,
            String format, double frameRate, int bitrate, String preset, int numBuffers) {
        try {
            ServerSocket server = new ServerSocket(port);
            Socket clientSocket = server.accept();
            FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(clientSocket.getInputStream());
            Java2DFrameConverter converter = new Java2DFrameConverter();
            grabber.setFrameRate(frameRate);// 60
            grabber.setFormat(format);// h264
            grabber.setVideoBitrate(bitrate);// 8000000
            grabber.setVideoOption("preset", preset);
            grabber.setNumBuffers(numBuffers);// 0
            grabber.start();// 阻塞,尝试过用线程开启(会编码失败)
            
            while (!Thread.interrupted()) {
                final Frame frame = grabber.grab();
                if (frame != null) {
                    final BufferedImage bufferedImage = converter.convert(frame);
                    ImageIO.write(bufferedImage, "png", new File("D://img//" + System.currentTimeMillis() + ".png"));
                    if (bufferedImage != null) {
                        Platform.runLater(() -> view.setImage(SwingFXUtils.toFXImage(bufferedImage, null)));
                    }
                }
            }
            grabber.close();
            server.close();
        } catch (final Exception e) {
            e.printStackTrace();
        }
    }
}

客户端Maven依赖

        <dependency>
            <groupId>org.bytedeco.javacpp-presets</groupId>
            <artifactId>ffmpeg-platform</artifactId>
            <version>4.0.2-1.4.3</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv</artifactId>
            <version>1.4.3</version>
        </dependency>

        <dependency>
            <groupId>org.jcodec</groupId>
            <artifactId>jcodec</artifactId>
            <version>0.2.3</version>
        </dependency>

        <dependency>
            <groupId>org.jcodec</groupId>
            <artifactId>jcodec-javase</artifactId>
            <version>0.2.3</version>
        </dependency>

Android端代码

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
    private boolean encode(MediaCodec codec, Socket fd) throws IOException {
        boolean eof = false;
        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

        while (!consumeRotationChange() && !eof && fd.isConnected() && !fd.isOutputShutdown() && !fd.isInputShutdown()) {
            int outputBufferId = codec.dequeueOutputBuffer(bufferInfo, -1);
            eof = (bufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0;
            try {
                if (consumeRotationChange()) {
                    // must restart encoding with new size
                    break;
                }
                if (outputBufferId >= 0) {
                    ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId);

                    if (sendFrameMeta) {
                        writeFrameMeta(fd, bufferInfo, codecBuffer.remaining());
                    }

                    IO.writeFully(fd.getOutputStream(), codecBuffer);
                }
            } finally {
                if (outputBufferId >= 0) {
                    codec.releaseOutputBuffer(outputBufferId, false);
                }
            }
        }

        return !eof;
    }

期望效果

怎样才能解决阻塞问题,当接收到H264流时就进行解码成BufferedImage,然后进行渲染操作。或者是否可以对字节数组进行单独解码成图片。

78KgMiao的主页 78KgMiao | 初学一级 | 园豆:169
提问于:2019-12-23 09:32
< >
分享
所有回答(1)
0

使用其他方式解决了该问题。参考github上一位大神的代码https://github.com/twilightdema/h264j。
[github地址]https://github.com/twilightdema/h264j

78KgMiao | 园豆:169 (初学一级) | 2019-12-24 15:10
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册