Loading...

orbited-users@googlegroups.com

[Prev] Thread [Next]  |  [Prev] Date [Next]

Re: [Orbited] Re: Orbited CPU usage keeps climbing Dmitry Utkin Tue Jun 29 01:01:21 2010

Niklas: haven't used 0.8 yet, in trac for this but i saw a milestone
0.7.1, but it never came to public, so it's hard to tell

On Tue, Jun 29, 2010 at 10:31 AM, Niklas B <[EMAIL PROTECTED]> wrote:
> Dmitry: Do you know if your issue is on Orbited 0.8 as well?
>
> tejas: using lsof -p <orbited pid>
>
> Do you see a lot of TCP connections to users that are never closed?
>
> On Jun 29, 9:25 am, Dmitry Utkin <[EMAIL PROTECTED]> wrote:
>> this can be this problem I've experienced 
>> recentlyhttp://groups.google.com/group/orbited-users/browse_thread/thread/c93...
>>
>> there is a patch by clemesha which helpedhttp://gist.github.com/256582
>>
>> Good luck!
>>
>>
>>
>> On Mon, Jun 28, 2010 at 5:30 PM, tejas <[EMAIL PROTECTED]> wrote:
>> > Yup - unclosed connections is the root cause. netstat reports 1025
>> > open connections to my https server, yet orbited's system monitor
>> > reports only ~50.
>>
>> > Any way to figure out which URLs are open? Or maybe it has something
>> > to do with keep-alives?
>>
>> > On Jun 20, 3:14 pm, tejas <[EMAIL PROTECTED]> wrote:
>> >> More information - looks like some connections arent getting closed:
>>
>> >> netstat -an | grep ":443" | grep "ESTABLISHED" | wc -l gives me ~1000,
>> >> though the system monitor reports only ~50.
>>
>> >> I also see a bunch of these in the logs:
>>
>> >> >> 06/19/10 13:00:48:944 WARN   orbited.proxy.ProxyIncomingProtocol       
>> >> >>  Connection Error [Failure instance: Traceback (failure with no 
>> >> >> frames): twisted.internet.error.ConnectBindError: Couldn't bind: 24: 
>> >> >> Too many open files.
>>
>> >> I assume thats because connections arent getting closed, and I had a
>> >> limit of max 1024 open files (I've now increased that limit to
>> >> 100000).
>>
>> >> Any idea how I can debug this further? I've just restarted orbited to
>> >> keep things working..
>>
>> >> On Jun 17, 7:28 pm, tejas <[EMAIL PROTECTED]> wrote:
>>
>> >> > I have nothing other than the HTTP, HTTPS and Monitor services
>> >> > running. Browsers connect to ActiveMQ through this Orbited instance.
>>
>> >> > Here's the config I use:
>> >> > [global]
>> >> > # reactor=select
>> >> > # reactor=kqueue
>> >> > reactor=epoll
>> >> > user = orbited
>> >> > proxy.enabled = 1
>> >> > session.ping_interval = 60
>> >> > session.ping_timeout = 50
>>
>> >> > [listen]http://[IPADDRESS]:80https://[IPADDRESS]:443
>> >> > monitor://:8005
>>
>> >> > [ssl]
>> >> > key=/path/to/private/key
>> >> > crt=/path/to/ssl/certificate
>>
>> >> > [static]
>>
>> >> > [access]
>> >> > * -> localhost:61613
>> >> > * -> subdomain.domain.com:61613
>> >> > * -> subdomain.domain.com:8005
>>
>> >> > [logging]
>> >> > debug=/var/log/orbited/debug.log
>> >> > info=/var/log/orbited/orbited.log
>> >> > access=/var/log/orbited/access.log
>> >> > warn=/var/log/orbited/orbited.log
>> >> > error=/var/log/orbited/orbited.log
>>
>> >> > #Don't enable debug by default
>> >> > enabled.default=info,access,warn,error
>>
>> >> > # Turn debug on for the "Proxy" logger
>> >> > [loggers]
>> >> > #Proxy=debug,info,access,warn,error
>>
>> >> > On Jun 16, 4:48 am, leon <[EMAIL PROTECTED]> wrote:
>>
>> >> > > I have Orbited running on a site for quite some time now - but I have
>> >> > > never had any memory issues (one of the processors keeps at 100% due
>> >> > > to my nasty SQL hammering the DB every second). It seems you're
>> >> > > loading something that is not getting released.
>>
>> >> > > On 15 jun, 02:18, tejas <[EMAIL PROTECTED]> wrote:
>>
>> >> > > > Hi - I have plugged in the orbited system monitor socket (port 8005)
>> >> > > > to ganglia so I have a high level view of how orbited has been
>> >> > > > performing over time. One thing I've noticed is the CPU always seems
>> >> > > > to be going up, even though the number of connections hover between 
>> >> > > > 20
>> >> > > > - 70.
>>
>> >> > > > Any idea why? I'm worried this might be an issue during scaling. I'm
>> >> > > > using v0.7.10
>>
>> >> > > > Take a look at these graphs for the last week:
>>
>> >> > > > Connections:http://public.bay.livefilestore.com/y1pm_Y0_ScAmu8tpEg1eZCIEJST4KKsy-...
>>
>> >> > > > CPU:http://public.bay.livefilestore.com/y1pm_Y0_ScAmu8FliK-mu_Z3Uv6rvxwY9...
>>
>> >> > > > Memory:http://public.bay.livefilestore.com/y1pejn3nhwFGYgw84sPKoVvy4Az8umxZq...
>>
>> > --
>> > You received this message because you are subscribed to the
>> > Orbited discussion group.
>> > To post, send email to
>> >    <[EMAIL PROTECTED]>
>> > To unsubscribe, send email to
>> >    <[EMAIL PROTECTED]>
>> > For more options, visit
>> >    <http://groups.google.com/group/orbited-users>
>
> --
> You received this message because you are subscribed to the
> Orbited discussion group.
> To post, send email to
>    <[EMAIL PROTECTED]>
> To unsubscribe, send email to
>    <[EMAIL PROTECTED]>
> For more options, visit
>    <http://groups.google.com/group/orbited-users>

-- 
You received this message because you are subscribed to the
Orbited discussion group.
To post, send email to
    <[EMAIL PROTECTED]>
To unsubscribe, send email to
    <[EMAIL PROTECTED]>
For more options, visit
    <http://groups.google.com/group/orbited-users>