On 3/22/07, philipp.marek@bmlv.gv.at <philipp.marek@bmlv.gv.at> wrote:
>
> > My goal was to recreate what Malcolm had done based solely on his
> > description above. But some of your ideas were interesting although they
> > don't appear match what Malcolm has done.
> Thank you.
>
> > OK, I did that. The summary: no change. NTFS appears to be just as fast
> for
> > this scenario as any other modern filesystem.
> >
> > Across 1024 file opens in a directory of 2^20 files, an average time
> > (seconds) of 0.025 for not-sharded and 0.026 for sharded. See the
> attached
> > scripts and log files for the details. I ran the read script three
> times,
> > recording the 3rd iteration's time to minimize the impact of my slower
> > laptop.
> Your script has 2^12, which is 4096, in it.
> Just out of interest: Could you post the complete log?
2^12 is the number of opens, not the number of files. Look at the last line
of the logs I posted earlier inside the zip file. That's where I copied the
those numbers from.
Also, you can also see from line 62 of the read script that the total time
is divided by the loop counter. Also, note the file names: these are created
by a loop counter so a filename of 922987.txt means that it was the
922,987th file to be created in that directory (for non-sharded, of course).
Dividing with 2^10 I get 25µs per open. I have seen completely different
> results
> ... which Windows is that? Are you sure that you really have 1 024 076
> files in
> this directory?
Yes, I'm certain there are 2^20 zero-byte files in the directory. Look at
line 9 of the create script. Also, it took roughly 10 minutes to create the
files and another 10 minutes to delete them. It doesn't take that long to
delete 1024 files. Looking at this directory in Explorer is painful. I tried
once and gave up. Thankfully svn doesn't have to enumerate the contents. I
too expected to see worse results than this. As was noted elsewhere, the
bottleneck must be Explorer and not NTFS.
Anyone else want to take a crack at running these scripts on NTFS to check
my numbers?
I'm running Windows XP Pro fully patched on a Dell Inspiron 8600 P-M
1.7GHzwith 2GB RAM and a 7200 100GB IDE hard disk. And I was using my
system for
other light-duty stuff like checking email, browsing the web, etc.
Received on Thu Mar 22 19:37:21 2007