Hi Johan.
I've just read the whole of this thread.
I didn't quite understand your original point (2) that "token-based
suffix scanning will not be as fast as byte-based suffix scanning".
Sure it won't, but is there any reason you mentioned suffix scanning
there specifically? The same is true of prefix scanning, of course.
And both of them could be fast enough, I assume, if you disable the hash
calculation in the "get token" callbacks like you were talking about.
But I don't think that necessarily affects the main point. It looks
like you've thoroughly investigated using a token based approach. Thank
you for doing so. My initial feeling that it was worth investigating
was in the hope that you might find some fairly straightforward and
self-contained modification to the existing token-handling layer. I
think the result of this investigation, in which you needed to add
token-fetch-backwards callbacks and so on, shows that this approach is
too complex. I don't want to see a complex implementation. Therefore I
support your inclination to abandon that approach and use the byte-wise
approach instead.
- Julian
Received on 2010-12-02 16:22:00 CET