[ILUG] Bigger pipes...
padraig at antefacto.com
Wed Apr 25 18:36:50 IST 2001
Kenn Humborg wrote:
> > On Wed, 25 Apr 2001, Paul Jakma wrote:
> > > > takes 20secs of CPU time. Doing them together takes 40secs,
> > > > while, in theory, the I/O and CPU could happen in parallel, giving
> > > > a run time of 20secs.
> > >
> > > hmm... usually i'd defer to your wisdom kenn, but in this case i can
> This'll learn you...
> > > not see how you think that you can reduce 20secs for reading, 20secs
> > > for gzipping to less than 20+20secs.
> > just to be clearer and avoid obvious retorts:
> > ...i do not see how you think you can get extra parallelism beyond any
> > you are getting already...
> And, in fact, mbuffer does the job nicely. The 40secs is currently
> reduced to 24secs. I use a dd to pull across NFS which feeds
> a nice big mbuffer process, which feeds gzip.
> Thanks, Padraig!
> BTW, Kevin, I already adjusted rsize & wsize to 8K.
why use dd? Doesn't that introduce another data copy?
couldn't you use mbuffer -i /images/test.img ?
Also completely guessing, mbuffer supports mmap which
might get rid of the output data copy also?
More information about the ILUG