On 05.02.2011 16:46, Stefan Sperling wrote:
> On Sat, Feb 05, 2011 at 04:22:29PM +0100, Stefan Küng wrote:
>> On 05.02.2011 13:56, Stefan Sperling wrote:
>>
>>> I think we should go into this direction.
>>> In fact, I think we should simply change the existing APIs to use
>>> the fastest possible way of getting at information.
>>
>> Well, currently there is no API that does what I suggested
>> (basically return all results of a db query without even touching
>> any files in the WC or have do this for every file/folder
>> separately).
>
> Well, the svn_proplist case I'm looking at is the same thing.
> I want to answer the request "Give me all properties within this
> working copy" by issuing as few sqlite queries as possible.
>
> You are talking about such requests in general.
> I am talking about one specific instance (proplist).
> But essentially we want the same thing.
Yes, the proplist case is what I need.
But of course, the other cases I listed are also valid requests. For
example "give me all entries that have the author XXX as the
last-commit" or "give me all paths that were changed since date XXX".
These could also be answered by sqlite queries alone.
Or even "give me all paths in this working copy" would also be a big
help. Getting that information without the sqlite db would require
crawling the whole working copy, resulting in a lot of disk access.
>> I've read up on that thread. It seems the problem you're facing
>> comes from the fact that you need to stay compatible with pre 1.7
>> APIs and clients, and the fact that you can't enforce clients to
>> behave, only to ask them to behave and then hope for the best.
>
> Yes.
>
>> However what I'm asking for here are *new* APIs which do something
>> no existing API currently does. So staying compatible wouldn't be a
>> problem.
>
> New APIs don't make the problems we have with existing APIs go away.
>
> The backwards compat problem doesn't affect TortoiseSVN, since
> you will simply provide builds linked to 1.7 and tell all users to upgrade.
>
> But we need to keep existing clients that were compiled against 1.6.x
> and earlier working. So it's not "not a problem", it's just not
> TortoiseSVN's problem :)
Sorry, I didn't realize the problem already exists. I was under the
impression that the new API that was reverted a day later only had that
problem and that's why it got reverted.
>> And if you're worried about clients not behaving properly,
>> why not get rid of the callback completely and just return all
>> information at once in one big chunk of memory.
>> Talking about UI clients, this won't be a problem because they
>> usually have to store all information they receive in a callback
>> anyway so they have it ready to show in the UI. So for them, the
>> memory use wouldn't be bigger at all.
>
> Really? Even with gigantic working copies?
> What if the amount of information requested simply doesn't fit
> into memory? I'd prefer a system that cannot fail in this way.
> I'd prefer passing the information to the client piece by piece,
> and letting the client worry about where to store it.
> If at all possible, the library shouldn't allocate huge slabs of
> memory outside of the client's control.
Sure, that would be the ideal way. But if it's not possible to force
clients to behave and if that's a requirement, allocating a lot of
memory might be the only alternative.
>> Of course, those APIs I'm asking for might not be very useful for
>> existing APIs or other stuff that is done in the svn library. Those
>> might only be useful for some svn clients. But I hope that's not a
>> blocker for implementing those.
>
> I hope that we'll get a good set of APIs for 1.7 that will
> satisfy all clients out there, including TortoiseSVN.
> What these APIs will look like isn't set in stone yet.
>
>> I also thought of just query the SQLite db myself directly, but then
>> I don't like to do something that's not really allowed.
>> However: I did a quick test with the Check-for-modifications dialog
>> in TSVN. It has a feature where you can enable showing all
>> properties. To do that, a separate thread is started which lists all
>> properties of all items in the working copy. On one of my working
>> copies, this takes about 50 seconds. Using a simple SQLite query on
>> the NODE table took in average 1260ms. Parsing the data and
>> preparing it for use in the UI took another 3.5 seconds. Now *that*
>> a speed improvement I really like.
>
> How is your query any different from the new proplist API and
> implementation I added in r1039808? I think that provides what you need
> (for the proplist case). It opens the db, runs a query on it and streams
> the results to the client via a callback. Very low overhead.
Haven't tried that proplist API. I just reviewed it and it really is
what I need. Too bad it got reverted. I hope it will get put back in soon.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net
Received on 2011-02-05 17:16:09 CET