Hi all.
I've been playing around with replacing the BDB stuff with equivalent
MySQL stuff, that is, I've:
- Copied the libsvn_fs/bdb directory to libsvn_fs/mysql.
- Modified the files of libsvn_fs/mysql to use MySQL queries to
store and retrieve data.
Bear in mind that, although they are currently compiling, things aren't
*working*. (I had forgotten to do anything with trail.c!) However I'm
at the point when I'd like some advice about how to continue. I'll
describe my thoughts and implementation. Please feel free to argue
about it.
I know Glenn Thompson is working on something
(http://www.cdrguys.com/subversion/sql_fs_docs/svn_fs_sql.htm and
http://www.cdrguys.com/subversion/fs_refactoring/index.html). But,
unlike the following, that seems to be quite an ambitious and serious
project.
Hopefully I'll get something *working* in the next 48 hours or so, and
I'll put the code up somewhere for other people to take away and
(optionally, but hopefully) improve.
I. Database structure
1. Keys for records: BDB generally uses base-36 strings for record
The keys, providing a ludicrous maximum number of possible keys.
obvious MySQL equivalent is an INT AUTO_INCREMENT field.
(Currently I just use a VARCHAR with a base-36 string, with the
next available key stored in a record called 'next-id', in the
first VARCHAR() field of that record. UGLY.)
2. Skels: Since MySQL allows more than one non-key column in a
table, I've done away with using skels. Data that is normally
stored in list skels now needs to reside in an additional table,
for instance, the 'representations' and 'transactions' tables.
(Currently I have added tables 'windows', 'transaction_props',
and 'transaction_copies' to contain that data. Today, I also
talked briefly with cmpilato on #svn about storing a TXN
reference in each copy, instead of using 'transaction_copies'.)
3. Strings: The BDB implementation stores strings as multiple
records under a single key. I'm not sure if this is necessary
for an SQL implementation. (Though I currently do the same
thing, except that each string record has additional 'from'
and 'to' fields that can be used to find appropriate string
records for a particular segment of a string.)
4. I'm using the InnoDB table type, which provides "transactions".
I personally haven't had much experience with transactional
databases, so I don't yet know if it'll be easy to provide the
same level of ACIDity as BDB has, but I have done some testing.
That is, I've tested how the 'next-id' record is used by
concurrent processses, and realised that "FOR UPDATE" must be
appended to SELECT queries that retrieve the next-id. This
probably wouldn't be necessary if AUTO_INCREMENTs were used;
but it might well be necessary for other records.
II. Code Integration
5. The MySQL-specific stuff is in a mysql/ subdirectory, in the
same vein as the bdb/ subdir. But there is still a lot of
BDB-specific stuff in fs.c and trail.c.
6. I haven't written code to track the memory allocated by the
MySQL functions yet. And it is possible that a lot more
memory is being used, since the SQL is generated and stored in
trail->pool (if the amount is significant I'll look into using
subpools).
7. I have read about G. Thompson's fs vtable work (though I
haven't seen it), and since it sounds like a good thing,
I've avoided duplicating the effort.
8. For the record, I'm also not sure how to translate the BDB error
macros to elegant MySQL equivalents...
III. The Repository
9. The MySQL database is stored outside the repository, and accessed
through sockets using the MySQL client library. Where should the
MySQL configuation data (hostname, username, password, database)
by stored, and what should be stored in place of the BDB data?
(Currently the MySQL data is stored in fs.c!) I'm inclined to
keeping the code to recognise a MySQL or BDB repository, and
creating/opening/etc'ing the database, in libsvn_fs.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Feb 19 11:56:12 2004