red5 out of memory exception

I’m trying to analyze heap dump of my application which is made when out of memory exception occurred.
I have video on demand application on red5 1.0.1 Final server and Flash client which is written in AS 3 and communicate with server through RTMPT

Why so much messages is accumulated in RTMPTConnection.pendingOutMessages ? See attachments for details.

any custom functions or config?

Piotr

no custom configs. What do you mean when talk about functions?

Well what does you app do exactly, perhaps you start some process that never finishes?

Can you post the custom configs (a diff file of compared to the originals) ? Now I’m not an expert on red5, but it would help to solve your issue by some of the smart folks in this group.

Cheers,

Piotr

May be this exceptions lead to memory leak? I found them in my log.
Also I found in profiler than org.red5.server.net.rtmpt.RTMPTConnection instances are never terminated after I close flash player or close browser tab. Their count grows.org.red5.server.net.protocol.ProtocolException: Error during decoding
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:195) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeBuffer(RTMPProtocolDecoder.java:116) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmpt.BaseRTMPTConnection.decode(BaseRTMPTConnection.java:195) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmpt.RTMPTServlet.handleSend(RTMPTServlet.java:336) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmpt.RTMPTServlet.service(RTMPTServlet.java:419) ~[red5-server-1.0.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) ~[javaee-api-5.1.2.jar:5.1.2]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:891) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:750) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2282) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
Caused by: java.lang.IllegalStateException: Invalid RTMP state: 5, nothing to decode
at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:189) ~[red5-server-1.0.jar:na]
… 21 common frames omitted

java.lang.NullPointerException: null
at org.red5.server.net.rtmpt.BaseRTMPTConnection.read(BaseRTMPTConnection.java:204) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmpt.RTMPTServlet.handleSend(RTMPTServlet.java:341) ~[red5-server-1.0.jar:na]
at org.red5.server.net.rtmpt.RTMPTServlet.service(RTMPTServlet.java:419) ~[red5-server-1.0.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) ~[javaee-api-5.1.2.jar:5.1.2]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) ~[catalina-6.0.36.jar:6.0.36]
at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:891) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:750) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2282) ~[tomcat-coyote-6.0.36.jar:6.0.36]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_25]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]

Between 1.0.1 and 1.0.2 (trunk) the encode/decode and connection handling were overhauled; I would suggest trying the latest red5 with your application as all known memory leaks have now been fixed.

This bug was reproduced on 1.0.2. I’ll try use version 1.0. It is likely more stable. Bug is reproduced after 3 hours of playing live video in flash player. Flash player was crashed after that (due to unknown issue) and soon OOM appears.

I can confirm this bug exists in red5 1.0.1, red5 1.0.2 and not exists in red5 1.0.0.

Reproducing steps:

1. Install demo version of YourKit java profiler.

2. Run red5 with command line arguments:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=”C:\path” -agentpath:”C:\Program Files\YourKit Java Profiler 2013 build 13052\bin\win32\yjpagent.dll”

3. Connect to your red5 with YourKit

4. Provide any live streming to red5 (I’ve used ffmpeg)

5. Start playing live stream in browser (I’ve used Chorme) from red5.

6. Suspend process of browser (I’ve used Sysinternals Process Explorer). That action results in adobe flash plugin crashes.

7. Look at Yourkit->Memory->Class list. Type RTMPTConnection in search string and push Refresh many times. You can see that size of RTMPTConnection.PendingData constantly increases.

I made 3 snapshots of memory: https://dl.dropboxusercontent.com/u/14792156/Out_of_memory_snapshots.7z

Screenshot of RTMPTConnection instance:

Red5 1.0.1 and Red5 1.0.2 after crash flash plugin starts accumulate data in RTMPTConnection.PendingData

Red5 1.0.0 accumulates data during several seconds after crash, but then the RTMPTConnection.PendingData growing stops

red5 out of memory exception

By Admin

Related Post

You Missed

Introduction to Secure RTMP Server Configuration

Setting up the Roku device and Roku SDK on an RTMP Server

RTMP Server in the era of HTTP video streaming

Ethiopia Web Hosting