[Rails] large file storing in postgres sucks?

[Joe Van Dyk]

On 4/30/06, Joe Van Dyk <joevandyk at gmail.com> wrote:
> On 4/30/06, Robby Russell <robby.lists at planetargon.com> wrote:
> >
> > On Apr 30, 2006, at 6:04 PM, Joe Van Dyk wrote:
> >
> > > Hi,
> > >
> > > I'm not sure if this is a postgres issue or not, but I'm using
> > > Postgres 8.1.  I have (in my functional tests) the following code:
> > >
> > >    upload = fixture_file_upload('/files/podcast.mp3', 'audio/mpeg')
> > >    post :create, :product => valid_product, :media => { :image =>
> > > upload }
> > >
> > > And then in the controller:
> > >  blob = Blob.create :data => uploaded_file.read
> > >
> > > When /files/podcast.mp3 is 32 megabytes and I run the test, memory
> > > spikes hugely.  My laptop quickly began swapping out memory and became
> > > unusable.  top showed the memory usage of the Ruby process to be about
> > > 400MB and growing.
> > >
> > > Is there something stupid that I'm doing?
> >
> > Are you using a blob field type rather than the postgresql large
> > objects?
>
> Probably.  Whatever using the :binary thingy in migrations gives me.

Even a 10 megabyte file makes the Rails process consume a hell of a
lot of memory.  And then postmaster consumes quite a lot as well.