[ILUG] Bigger pipes...

[Padraig Brady]

Kenn Humborg wrote:
> 
> > On Wed, 25 Apr 2001, Paul Jakma wrote:
> >
> > > > takes 20secs of CPU time.  Doing them together takes 40secs,
> > > > while, in theory, the I/O and CPU could happen in parallel, giving
> > > > a run time of 20secs.
> > >
> > > hmm... usually i'd defer to your wisdom kenn, but in this case i can
> 
> This'll learn you...
> 
> > > not see how you think that you can reduce 20secs for reading, 20secs
> > > for gzipping to less than 20+20secs.
> >
> > just to be clearer and avoid obvious retorts:
> >
> > ...i do not see how you think you can get extra parallelism beyond any
> > you are getting already...
> 

[snip]

> And, in fact, mbuffer does the job nicely.  The 40secs is currently
> reduced to 24secs.  I use a dd to pull across NFS which feeds
> a nice big mbuffer process, which feeds gzip.
> 
> Thanks, Padraig!
> 
> BTW, Kevin, I already adjusted rsize & wsize to 8K.
> 
> Later,
> Kenn

why use dd? Doesn't that introduce another data copy?
couldn't you use mbuffer -i /images/test.img ?
Also completely guessing, mbuffer supports mmap which
might get rid of the output data copy also?

Padraig.