[Rails] RubyGems runtime memory use in Rails apps...
Steve Longdo
steve.longdo at gmail.com
Sat Aug 12 07:47:10 GMT 2006
I have posted an update on this at my blog:
http://www.stevelongdo.com/articles/2006/08/12/rubygems-memory-reduction
Basically RubyGems needs to be patched. Stef's solution below works
perfectly, once you figure out how to set it up on a shared host! I will be
following up on exactly how to do that tomorrow.
Looks like maybe Nicholas Seckar and the Rails core are going to be doing
something different soon anyway.
http://weblog.rubyonrails.org/2006/8/11/reloading-revamped
On 8/9/06, Stef T <stef at ummon.com> wrote:
>
>
> Hello Steve,
> No no, your definitely not a pariah, and your emails are definitely
> making it onto the list ;) A crazy thought appears, perhaps woefully and
> stupidly in my head, but, its 1AM EST here, so, forgive me for this,
> but, why not simply use the design pattern of Singleton in
> custom_require.rb if you believe this is the problem ?
>
> There now follows a rather clunky 'human diff' .. please bear with me
> :)
>
> at line 24 (in the require method)
> @gempath_searcher ||= Gem::GemPathSearcher.new
> should become
> @gempath_searcher = Gem::GemPathSearcher.instance
>
> And then you want to change the module Gem part below to look
> something like;
>
> module Gem
> require 'singleton'
> #
> # GemPathSearcher has the capability to find loadable files inside
> # gems. It generates data up front to speed up searches later.
> #
> class GemPathSearcher
> include Singleton
>
> If you then put a 'puts @gempath_searcher.object_id' after the line
> 24 change, and run (say) your ruby ./script/server, you will see a huge
> spew of lines like ;
>
> stef at stef-desktop:~/work/myTest$ ruby ./script/server
> -606820690
> -606820690
> -606820690
>
> If you 'undo' those changes, and go back to the good old-fashioned
> code, you will see something like;
>
> stef at stef-desktop:~/work/myTest$ ruby ./script/server
> -606681356
> -606869086
> -606681356
> -606818976
>
> Different object_id's means different object's .. so .. by using the
> Singleton design pattern, your gemserver will now only be created
> 'once'. Now, of course, wether or not this breaks things horrible OR if
> this does indeed fix your memory leakage, you will have to let us all
> know :)
>
> Either way, have a nice night, I am shuffling off to bed, for some
> well earned R&R. Let me know how it all works out though please,
> irregardless ;)
> Regards
> Stef
> (ps. I could never worship Satan, but only Cthulhu, mean to say, why
> worship a lesser evil ;D
> (pps. that -is- a joke for the humour impaired ;)
> Steve Longdo wrote:
> > Stef!!!!
> >
> > Thank you for answering at all! I am a leper on the list it feels
> > like sometimes. I am profiling Typo, an app in need of some very
> > "Mature Optimization". So no worries there. Locally I use OS X/PPC,
> > Ruby 1.8.4. However I am doing the real profiling against my
> > www.stevelongdo.com <http://www.stevelongdo.com> site on TextDrive.
> > It is also Ruby 1.8.4 and I think a Linux flavor not Solaris 10 yet
> > anyway.
> >
> > Typo and its external dependant version of Rails were trunk a couple
> > days ago. TextDrive limits are based on RSZ going over 48MB. VSZ is
> > almost always lower than that when Ruby decides to allocate some more
> > space and send RSZ over the limit. I am not sure whether they are
> > using Stefan Kaes suggested patched GC or if it is plain vanilla Ruby
> > GC. I don't for sure how to find that out on TextDrive.
> >
> > At any rate there is no good reason for my personal Typo weblog to
> > cross into the 48MB zone. I'll bet all of my content together is 512kB.
> >
> > Outside of all that though I am trying to learn more about how
> > RubyGems actually does its business and whether it could do it in less
> > than ~11MB of RAM. I am barreling into the code now since response to
> > my "plea" for help went unanswered. I would expect that Shared Host
> > providers would want to have somebody look at this because it should
> > be an issue for all Rails apps not just Typo. Imagine if we figure
> > out how to shave 10MB off of each Rails app out there? Shared Hosts
> > become more viable, people send me presents and bow in my presence...
> >
> > I've gotten a bit further in my research tonight take a look at my
> > latest post:
> >
> http://www.stevelongdo.com/articles/2006/08/09/rubygems-single-multi-tons-and-memorysize
> > <
> http://www.stevelongdo.com/articles/2006/08/09/rubygems-single-multi-tons-and-memorysize
> >
> >
> >
> > All hail Stef our dark underlord...Oh not affiliated with Satan, I
> > see. :-P
> >
> > On 8/8/06, *Stef T* <stef at ummon.com <mailto:stef at ummon.com>> wrote:
> >
> >
> > Hello Steve,
> > Well, since you asked for help, can I suggest that you provide
> > pertinent details ? OS used, ruby version, and also the example code
> > that you used to profile would be nice ;) I must say, that rubygems
> > hooks into the main process from what I can (hence the require) so I
> > would -expect- it to stay resident in memory. Which reminds me, what
> > memory are you profiling here ? VSZ ? RSZ ? Then again, in the day
> and
> > age of 1GB in my desktop machine, its probably highly easy to
> > dismiss a
> > RSZ of 10MB ;) Heck, almost all of the servers at work have >4GB.
> > Thats
> > not the point, I know, but, there is no point in chasing rabbits
> down
> > holes if you don't need to (also called the 'premature optimisation'
> > problem).
> >
> > Details details, the devil is in the details*
> > Regards
> > Stef
> > (* this should not be taken as an endorsement of satan nor that I
> have
> > -the- answer to your problems ;)
> >
> > Steve Longdo wrote:
> > > This an open plea for someone with knowledge of RubyGems to
> explain
> > > why RubyGems stays in memory (over 10MB) after a Rails
> > application is
> > > loaded. I am unable to figure out why this would be necessary. I
> > > haven't had any response from Jim or Chad from RubyGems about it
> > so I
> > > thought I would throw it open to the list to see if anyone could
> > > educate me or share thier experiences with this.
> > >
> > > There is a more detailed write up on my blog (
> > >
> >
> http://www.stevelongdo.com/articles/2006/08/08/rubygems-dependency-management-and-memory-use
> > >
> > <
> http://www.stevelongdo.com/articles/2006/08/08/rubygems-dependency-management-and-memory-use
> > <
> http://www.stevelongdo.com/articles/2006/08/08/rubygems-dependency-management-and-memory-use
> >>
> > > ).
> > >
> >
> > -
> > Thanks,
> > -Steve
> > http://www.stevelongdo.com
>
>
--
Thanks,
-Steve
http://www.stevelongdo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://wrath.rubyonrails.org/pipermail/rails/attachments/20060812/c243a7d9/attachment-0001.html
More information about the Rails
mailing list