Erik and I chatted on IM about the NODE_DATA table (aka "4th tree"). Figured
it would be a good thing to capture that here to the dev@ list. Below is our
chat, with only a few (personal) redactions.
There is some more conversation, which I'll forward separately...
Cheers,
-g
---------- Forwarded message ----------
From: Erik Huelsmann
Date: Mon, Jul 5, 2010 at 16:38
Subject: Chat with Erik Huelsmann
To: gstein_at_gmail.com
[...]
Erik: From the conversation the past weeks, I see there are 3 big items
remaining for 1.7; one of which is the "4th tree"
15:45 me: yup
Erik: I was pondering the subject, but thought I'd tell you where my sudden
interest comes from before jumping into the porcelain
me: hehe
search your archives for NODE_DATA
that'll provide msot of the basics of the thinking
15:46 my last note said something about including copyfrom data in the
NODE_DATA table, but I'm nuking that idea
Erik: I read up on it this afternoon, or at least quite a bit of it.
ok.
I think there are 2 ways of viewing NODE_DATA:
1. as a stash for 'non-current' layers
15:47 2. as a table which holds all layers including the current one
from what I read about it, your thinking is (2)?
me: yes
the data moves from both BASE_NODE and WORKING_NODE into that table,
15:48 and with particular queries, you get the "latest" node of the tree
latest/most-current/topmost-layer/whatever
Erik: k. if we make mistakes clearing out the table, that's probably the
best way to notice early :-)
15:49 me: :-)
15:50 Erik: About using it for BASE_NODE as well as WORKING: there's no
intention to share records between copied parts of the tree though, right?
I mean, it'll all still be keyed on the local_relpath
me: correct
Erik: ok. because if not, I was expecting issues with e.g. presence
me: yah. not even gonna try that. rows will be copied when a copy/move
occurs.
15:51 Erik: Ok. Mind me writing up some of the thoughts in a mail? It should
end up being a proposal for change of the current schema.
15:52 me: please go ahead! sure
Erik: How far away do you expect yourself to be from moving to something
other than cleaning svn_wc__get_entry()?
15:53 me: I've got a few days of writing in-db props tests, then to bump
that format,
then to work on NODE_DATA
15:54 one complication is entries upgrading,
we have no more entry_modify() calls,
but we still have to write old entries into the db,
and that is done (today) using sql statements,
which will need to switch over to updating NODE_DATA as appropraite
15:56 Erik: ok. and from Bert, I understood that's the same time when we get
feature parity with 1.6 (ie being able to replace parts of the tree)
me: yah. we have a couple problems with adds-under-copy. a couple other
sequences.
15:57 Erik: philip was expecting problems from the "multi-copysource"
paradigm used to model mixed-rev WCs.
especially because you can't tell they're part of the same op.
do you see that differently?
15:58 me: I had thought to put the copyfrom_relpath/rev into the NODE_DATa
table to do mixed revs under one op,
but am going back on that idea,
and sticking to multiple ops in WORKING_NODE,
15:59 where each op specifies a different rev,
and yes... that will cause problems to detect "single op",
but that model is what we need for *commit* time,
Erik: the question should probably be "do we need to".
right.
me: because when committing, we issue a new COPY for each operation in the
WORKING table,
16:00 and so... yah. "fine. it looks like different ops.",
but do we care?
it is entirely possible that a person ended up in that state with TRUE
multiple operations,
or it is possible to reach taht state from a single mixed-rev copy,
Erik: when looking at it from a commit point of view, everything is part of
the same -yet uncreated - transaction, I guess.
me: and I think it is important to NOT contain that kind of history,
yes
but the biggest user-visible feature,
is "revert",
16:01 because you can only revert at operation roots,
not children of those,
so a mixed-rev copy will create multiple operation roots,
which can then be independently reverted,
but this can cause a problem because the ancestor node that has a
different revision,
Erik: do we need elision for those? What if everything is updated to the
same rev?
me: doesn't have the now-reverted descendent at that ancestor revision,
16:02 so when reverting in this situation, and there are no other layers of
NODE_DATA to provide the data,
then you have to mark the node as excluded,
so parent is r5, child is r7, and you revert the child,
it now becomes an r5/excluded child,
but even then... that might not be quite right because the child was
created in r6,
16:03 so maybe it just becomes a not-present node...
Erik: what happens to the excluded/not-present nodes during a commit? Do
they get copied with the parent, if in the repos? Or are they deleted, if
present in the repos?
me: post-copy, an update will not unify the revisions. they are still
copies of distinct revisions
copied with the parent
16:04 consider two operations: svn cp A newA ; svn cp B newA/B
well... insert an 'svn rm newA/B' in between
16:05 if newA/B is reverted, then the commit has a copy of A including all
of its children
Erik: right.
me: now... if you reach that same state via a single mixed-rev copy, then
you revert the newA/B "copy", then a commit should contain all of newA
(well, as a copy of A_at_some-rev)
16:06 iow, history says whether you have child data after that child-revert,
and you don't in a mixed-rev copy,
so you have to leave something there. I think that is not-present
16:07 Erik: it's not excluded (because that assumes existence) or deleted
(same reason)
16:08 so, you need something which says "it might or might not be here in
the repos, but I don't have it"
me: which is not-present. we report not-present to the server, and it will
send stuff if something should be there. or it will NOT send something, and
we remove the not-present node in that case.
Erik: sounds like absent, although currently that may assume existence too.
me: no. absent is a misnomer for not-authorized.
16:09 that's on a todo list for renaming.
16:10 Erik: ok. I get the context. Let's see if I can get something in a
mail about it.
me: not-present basically means "this is a versioned node at
*some*revision. we don't have its details here right now."
16:11 it primarily appears when you commit the deletion of a file. the
parent's rev implies its existence, but post-commit-rev it does not exist.
the parent has to have some kind of marker about the child, and that is
not-present
16:12 Erik: hmm. does our editor have this notion?
16:13 me: the not-present concept is part of the Reporter, not the Editor
we report not-present nodes to the server. when the server gives us stuff
to update our working copy, the editor will put a node there, or say nothing
about it.
16:15 (and as I said, post-update, we remove all not-present nodes; if the
server said nothing about them, then they are not part of the target
revision, so we can safely remove them)
now... part of the issue is that we're talking about copy/move children,
16:16 rather than nodes living in the BASE tree,
so I'd advise creating a new presence for these, for clarity sake.
in fact, during a conversation at some point, I suggested expanding the
set of presence values. include copy-here/move-here/moved-away into the
present set,
I erroneously thought it Good to keep a minimal set,
16:17 but that isn't really true. and it means we need to do a
scan_addition/scan_deletion to get data,
when we may only need the status that is easily derived from the presence
value
(obviously, you always have to scan for an operation root; tho... with
NODE_DATA and the op_depth, the scan is more of a *skip* :-) )
16:18 Erik: sounds sane.
we might want to remove the scans by expanding the presence set if
NODE_DATA is going quickly enough.
16:19 [...]
me: yeah
Erik: but it would be nice to carve out NODE_DATA before then.
[...]
me: anyways. yeah. in the course of adding NODE_DATA, then we can also
expand the set of presence values to assist with various types of data
lookups
16:26 (scanning will still be necessary, but we may be able to improve the
algorithm)
16:28 Erik: to recap: NODE_DATA contains the information to link a BASE_NODE
or WORKING_NODE to its repository location.
me: no
16:29 only BASE nodes have repository locations.
the table has about eight columns. I listed those out in one of the
emails.
16:31 kind, [checksum], changed_*, properties, [symlink_target]
7 columns of data. and then the key is <wc_id, local_relpath, op_depth>
16:32 we may also want to put translated_size and last_mod_time into
NODE_DATA
s/may/probably/
Received on 2010-07-05 22:59:01 CEST