[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Changes in SQL backend design

From: Jan Horák <horak.honza_at_gmail.com>
Date: Tue, 06 Apr 2010 13:58:29 +0200

Dne 6.4.2010 10:50, Philipp Marek napsal(a):
> Hello Jan!
> On Dienstag, 6. April 2010, Jan Horák wrote:
>> Hi, I've made some changes in SQL backend desing, updated design is at:
>> http://www.stud.fit.vutbr.cz/~xhorak50/diplomathesis/files/100405_mysql_des
>> ign_v2.png
> ...
>> The last thing is the discussed storing of the representations in files,
>> not in DB. I've made a simple test of the access speed to database and
>> to pure filesystem, the report is here:
>> http://www.stud.fit.vutbr.cz/~xhorak50/diplomathesis/files/100406_data_acce
>> ss_speed_comparison.pdf
> Fine, thank you very much.
>> The conclusion is, that SQL can be faster when many small files are
>> readed the first time (thanks better cache), in other cases the
>> filesystem is faster, as supposed. But Greg S. had a very good point,
>> that I agree with:
>> > My gut says "not that much faster". In most scenarios, the network
>> > bandwidth between the client/server will be the bottleneck. Reading
>> > the data off a disk (rather than from a DB) is not going to make the
>> > WAN connection any faster.
>> ...
>> > I'd go with the "store content in the database" until performance
>> > fiures (or a DBA) demonstrates it is a problem.
>> Thus the data stay in DB right now.
> I'm fine with that.
> Just a minor nit: how about allowing a mixed design (later, optional, when the
> backend is running)? Ie keep blocks smaller than N in the database, but write
> larger ones to the filesystem? Or provide different paths depending on the
> block size?
> Then people with SSDs could use them for the small blocks (or just keep them
> in the database, as before), but larger data entities could be read from disk
> directly.
> That would probably make some sense, as small blocks don't matter if they're
> travelling across a few pipes; but for multi-MB data blocks that should be
> avoided.

This is interesting idea and this approach seems to be the fastest from
my perspective.

On the other hand it would be quite complicated and non-complex. I would
rather wait until some SQL
backend prototype will exist, then try all possible alternatives and
choose the best one.


Jan Horak
Received on 2010-04-06 13:59:06 CEST

This is an archived mail posted to the Subversion Dev mailing list.