Edmund, I think it's only fair to tell you now: this couldn't be
integrated into Subversion unless it takes a similar route to what
Glenn Thompson is doing, whereby he first abstracts the fs APIs and
then implements the new backend against the APIs. (Of course, in that
case, it might just be better to join forces with Glenn.)
You probably knew already all this, and chose to copy the bdb/ subdir
and hack away because you wanted to do fast experimenting, without any
intention for it to get integrated back into Subversion. This mail is
just in case that's *not* what you were thinking...
Best,
-Karl
Edmund Horner <chrysophylax@chrysophylax.cjb.net> writes:
> I've been playing around with replacing the BDB stuff with equivalent
> MySQL stuff, that is, I've:
>
> - Copied the libsvn_fs/bdb directory to libsvn_fs/mysql.
> - Modified the files of libsvn_fs/mysql to use MySQL queries to
> store and retrieve data.
>
> Bear in mind that, although they are currently compiling, things aren't
> *working*. (I had forgotten to do anything with trail.c!) However I'm
> at the point when I'd like some advice about how to continue. I'll
> describe my thoughts and implementation. Please feel free to argue
> about it.
>
> I know Glenn Thompson is working on something
> (http://www.cdrguys.com/subversion/sql_fs_docs/svn_fs_sql.htm and
> http://www.cdrguys.com/subversion/fs_refactoring/index.html). But,
> unlike the following, that seems to be quite an ambitious and serious
> project.
>
> Hopefully I'll get something *working* in the next 48 hours or so, and
> I'll put the code up somewhere for other people to take away and
> (optionally, but hopefully) improve.
>
>
> I. Database structure
>
> 1. Keys for records: BDB generally uses base-36 strings for record
> The keys, providing a ludicrous maximum number of possible keys.
> obvious MySQL equivalent is an INT AUTO_INCREMENT field.
> (Currently I just use a VARCHAR with a base-36 string, with the
> next available key stored in a record called 'next-id', in the
> first VARCHAR() field of that record. UGLY.)
>
> 2. Skels: Since MySQL allows more than one non-key column in a
> table, I've done away with using skels. Data that is normally
> stored in list skels now needs to reside in an additional table,
> for instance, the 'representations' and 'transactions' tables.
> (Currently I have added tables 'windows', 'transaction_props',
> and 'transaction_copies' to contain that data. Today, I also
> talked briefly with cmpilato on #svn about storing a TXN
> reference in each copy, instead of using 'transaction_copies'.)
>
> 3. Strings: The BDB implementation stores strings as multiple
> records under a single key. I'm not sure if this is necessary
> for an SQL implementation. (Though I currently do the same
> thing, except that each string record has additional 'from'
> and 'to' fields that can be used to find appropriate string
> records for a particular segment of a string.)
>
> 4. I'm using the InnoDB table type, which provides "transactions".
> I personally haven't had much experience with transactional
> databases, so I don't yet know if it'll be easy to provide the
> same level of ACIDity as BDB has, but I have done some testing.
> That is, I've tested how the 'next-id' record is used by
> concurrent processses, and realised that "FOR UPDATE" must be
> appended to SELECT queries that retrieve the next-id. This
> probably wouldn't be necessary if AUTO_INCREMENTs were used;
> but it might well be necessary for other records.
>
>
> II. Code Integration
>
> 5. The MySQL-specific stuff is in a mysql/ subdirectory, in the
> same vein as the bdb/ subdir. But there is still a lot of
> BDB-specific stuff in fs.c and trail.c.
>
> 6. I haven't written code to track the memory allocated by the
> MySQL functions yet. And it is possible that a lot more
> memory is being used, since the SQL is generated and stored in
> trail->pool (if the amount is significant I'll look into using
> subpools).
>
> 7. I have read about G. Thompson's fs vtable work (though I
> haven't seen it), and since it sounds like a good thing,
> I've avoided duplicating the effort.
>
> 8. For the record, I'm also not sure how to translate the BDB error
> macros to elegant MySQL equivalents...
>
>
> III. The Repository
>
> 9. The MySQL database is stored outside the repository, and accessed
> through sockets using the MySQL client library. Where should the
> MySQL configuation data (hostname, username, password, database)
> by stored, and what should be stored in place of the BDB data?
> (Currently the MySQL data is stored in fs.c!) I'm inclined to
> keeping the code to recognise a MySQL or BDB repository, and
> creating/opening/etc'ing the database, in libsvn_fs.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Feb 19 18:03:15 2004