A Modest Proposal for Subversion Shared Files Within the Repository ------------------------------------------------------------------- ***** ROUGH DRAFT 1 ****** Subversion 1.1.x ****** 2004-02-19 ***** DISCLAIMER: No papist infants were eaten nor harmed in the production of this document. JUSTIFICATION: -------------- On the main Subversion page at http://subversion.tigris.org/, one of the features planned for post-1.0 is that Subversion: "...will handle symbolic links ("shortcuts"). It may also support multiple hard links and other special file types, as long as this can be done portably and with semantics that are compatible with version control." Another justification is user demand, and appearance in some other SCM systems. Many developers, including myself, have a need to have files shared across different parts of the same repository. Whenever one of the shares is updated, all others automatically update as well. SCOPE: ------ This proposal shall cover only repository-side links or "shares". It shall not endeavor to try to make client-side symbolic links. For sake of clarity, this document will refer to this repository-side linking as "sharing" so as not to be confused with filesystem links. TERMS: ------ root share : Primary shared file. This is the only node that contains a text-base secondary share : Any other shared node other than the root node. REQUIREMENTS: ------------- 1) Must be able to share and unshare from the svn client. 2) Must be able to determine if a file is a share from the svn client. 3) Accessing the share from WC shall act no differently than accessing any other WC file. 3a) On checkout or update, every share shall generate the same file on the client side. 3b) svn status shall show attributes just as if it were a normal file. 4) Upon checkin of only one of the shares, the repository will automatically update and create new revisions of all other shared nodes. 5) Upon commit of more than one linked shares, the files must be merged before commit to the root share node. 5a) Successful merge shall result in success 5b) Conflict during merge shall result in failure of commit. Example SIMPLE repository: -------------------------- Types: D == Dir, F == File, S == Share (Really of File type, but no text-base) Rev: Rev is the HEAD rev for that file Rev File Type ShareList 1 /prj1 D -- 2 /prj1/f1 F 3 /prj1/f2 F 1 /prj2 D -- IMPLEMENTATION: --------------- 1) Only one copy of a share shall be stored in the repository. All other shares (secondary) will be empty file nodes with no text-base. 2) When the repos gets a request to update a share, it finds the master share and updates it. Then all secondary nodes get a new copy, and the repos updates all nodes with the new shared node list (see #3 following) 3) Each share in a group of shared files shall have a list of all other related shared nodes, so that upon update, all other shared nodes may be updated to the new root node. 3a) The root share shall always be first in the list 3b) Sharing a file to a directory that doesn't exist is an error. Ex 3.1: svn share /prj1/f1 /prj2/f1 (Rev 4) Rev File Type ShareList 1 /prj1 D -- 4 /prj1/f1 F,S (/prj1/f1:4 /prj2/f1:4) 3 /prj1/f2 F 1 /prj2 D -- 4 /prj2/f1 S (/prj1/f1:4 /prj2/f1:4) Ex 3.2: svn share /proj1/f1 /proj3/f1 --> ERROR Add more shares to the share group: Ex 3.3: svn mkdir /proj3 (Rev 5) svn share /proj1/f1 /proj3/ghudson (Rev 6) [This rev change not shown] svn share /proj1/f1 /proj1/sussman (Rev 7) Rev File Type ShareList 1 /prj1 D -- 7 /prj1/f1 F,S (/prj1/f1:7 /prj2/f1:7 /prj3/ghudson:7 /prj1/sussman:7) 3 /prj1/f2 F 7 /prj1/sussman S (/prj1/f1:7 /prj2/f1:7 /prj3/ghudson:7 /prj1/sussman:7) 1 /prj2 D -- 4 /prj2/f1 S (/prj1/f1:7 /prj2/f1:7 /prj3/ghudson:7 /prj1/sussman:7) 5 /prj3 D -- 7 /prj3/ghudson S (/prj1/f1:7 /prj2/f1:7 /prj3/ghudson:7 /prj1/sussman:7) 4) Upon the breaking of a secondary share from a root share, the secondary will be updated to a full file status with a text base and no shares. It will be labeled as a copy of the root share. All other existing shares will be updated as removing the broken share from their list of shared nodes. Ex 4.1: svn unshare /prj3/ghudson (Rev 8) Rev File Type ShareList 1 /prj1 D -- 8 /prj1/f1 F,S (/prj1/f1:8 /prj2/f1:8 /prj1/sussman:8) 3 /prj1/f2 F 8 /prj1/sussman S (/prj1/f1:8 /prj2/f1:8 /prj1/sussman:8) 1 /prj2 D -- 8 /prj2/f1 S (/prj1/f1:8 /prj2/f1:8 /prj1/sussman:8) 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 5) Upon the breaking of a root share, the first secondary share will be promoted to the root, and given a text base, and all other shares updated appropriately. The new root node shall be labeled as a copy of the old root node. Ex 5.1: svn unshare /prj1/f1 (Rev 9) Rev File Type ShareList 1 /prj1 D -- 9 /prj1/f1 F 3 /prj1/f2 F 9 /prj1/sussman S (/prj2/f1:9 /prj1/sussman:9) 1 /prj2 D -- 9 /prj2/f1 F,S (/prj2/f1:9 /prj1/sussman:9) [Copy of /prj/f1:8] 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 6) Upon checkin, all shares within commit are merged together. If a conflict results, the commit shall be aborted. If only one share is affected by the commit, then no merge is necessary, but all shared nodes become part of the commit, as usual. 6a) Upon success, the client shall be notified of all shared nodes participating in a commit, so that it may optionally update its working copies of the shared files. Ex 6.1: svn ci / (Rev 10) M /prj1/sussman M /prj2/f1 Rev File Type ShareList 1 /prj1 D -- 9 /prj1/f1 F 3 /prj1/f2 F 10 /prj1/sussman S (/prj2/f1:10 /prj1/sussman:10) 1 /prj2 D -- 10 /prj2/f1 F,S (/prj2/f1:10 /prj1/sussman:10) [Merge of above shares] 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 7) When the repos gets a request to return text of a share, it looks in the list of shared nodes and finds the master (always first) and returns its contents. The repos shall also notify the client that it is a shared node. 8) Any commits on a share must verify that it is in the list of shares. If not, it will be converted to a file and unshared automatically. Ex 8.1: If a directory is copied which contains a share, you will have dangling share references. (Thanks to ghudson for bringing this up) svn cp /prj1 /prj4 (Rev 11) Rev File Type ShareList 1 /prj1 D -- 9 /prj1/f1 F 3 /prj1/f2 F 10 /prj1/sussman S (/prj2/f1:10 /prj1/sussman:10) 1 /prj2 D -- 10 /prj2/f1 F,S (/prj2/f1:10 /prj1/sussman:10) 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 11 /prj4 D -- 11 /prj4/f1 F [Node points to /prj1/f1:9 - cheap copy] 11 /prj4/f2 F [Node points to /prj1/f2:3 - cheap copy] 11 /prj4/sussman F,S (/prj2/f1:10 /prj1/sussman:10) [Node points to /prj1/sussman:10 - cheap copy] Then a user updates /prj1/sussman: (Rev 12) Rev File Type ShareList 1 /prj1 D -- 9 /prj1/f1 F 3 /prj1/f2 F 12 /prj1/sussman S (/prj2/f1:12 /prj1/sussman:12) 1 /prj2 D -- 12 /prj2/f1 F,S (/prj2/f1:12 /prj1/sussman:12) 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 11 /prj4 D -- 11 /prj4/f1 F [Node points to /prj1/f1:9 - cheap copy] 11 /prj4/f2 F [Node points to /prj1/f2:3 - cheap copy] 11 /prj4/sussman F,S (/prj2/f1:10 /prj1/sussman:10) [Node points to /prj2/f1:10 - cheap copy] If another user updates or checks out /prj4/sussman, we are still fine, because it is still pointing internally /prj2/f1:10 and that node hasn't changed. Now, a user updates /prj4/sussman. We must finally clean up the node and convert it. We see that /prj4/sussman is NOT in the node list. We must immediately copy the root of the share (/prj2/f1:10), and merge it with the update to make the new node. svn ci /prj4/f1 (Rev 13) Rev File Type ShareList 1 /prj1 D -- 9 /prj1/f1 F 3 /prj1/f2 F 12 /prj1/sussman S (/prj2/f1:12 /prj1/sussman:12) 1 /prj2 D -- 12 /prj2/f1 F,S (/prj2/f1:12 /prj1/sussman:12) 5 /prj3 D -- 8 /prj3/ghudson F [Copy of /prj1/f1:7] 11 /prj4 D -- 11 /prj4/f1 F [Node points to /prj1/f1:9 - cheap copy] 11 /prj4/f2 F [Node points to /prj1/f2:3 - cheap copy] 13 /prj4/sussman F PROBLEMS: --------- Q: On commit of one or more shared files, how do the uncommited shared files get updated in the WC without needing a svn up or svn co? A: The repository sends back a list of all shared nodes in a commit to client. It is the client's responsibility to update WC versions of the shares.