this time it seems that the script worked properly, even for duplicates linked together

Cette fois-ci, je perds 111.703 - 110755 = 948 (0,85%)
Tandis que le français perd 248.522 - 245.926 = 2.596 (1,04%)

C'est un peu plus cohérent car, cette fois-ci, je ne perds pas davantage de phrases que le corpus français !

Désolé d'avoir produit un peu moins de 1% de doublons, mais vu la quantité de contributeurs et le nombre de cas disjoints (en anglais : it's, this is, that is, that's, ...) je reste assez fier de mes résultats.

Or l'espéranto passe de 362.338 à 353.058, soit une diminution de 2,56%
L'italien passe de 284.911 à 277.499, soit une diminution de 2,60%
Le russe passe de 222.448 à 215185, soit une diminution de 3,26%

C'est en phase avec les annonces hebdomadaires de CK.
On n'est pas à l'abri de cas individuels, mais, statistiquement, ça ma paraît tenir la route et je tamponne donc !

I see some problems, mainly in the logs.

1. According to the log, many sentences are linked to itself (when duplicates had been linked to each other). For example, http://dev.tatoeba.org/eng/sentences/show/1179555 is "linked to #1179555", which is obviously strange.

2. When deleting a sentence, Horus pretends that he only deletes a sentence, while he actually deletes a sentence and links. As a result, the number of "linked to" minus the number of "unlinked from" in the log doesn't match the number of links. Again, this seems to happen only when duplicates had been linked to each other. Maybe it's better to delete the logs about these links between duplicates altogether.

3. Horus also doesn't tell us when he links sentences. So when we, for example, see the log of http://dev.tatoeba.org/eng/sentences/show/486749, it looks as if it's still linked to #3466390, even though in fact Horus deleted this link and linked the sentence to #2178443 instead. Of course #3466390 and #2178443 are the same thing, but I think it's a little problematic that we can't easily know who linked the sentence to #2178443 and which sentence marafon linked to (no sentence is highlighted when I click on "linked to #3466390".)
However, I'm not sure what's the best thing to do. I'm tempted to simply change the log to "marafon - Sep 2nd 2014, 12:52 linked to #2178443", but we shouldn't do this. When a sentence has been changed, falsification of logs can cause serious problems.

4. Do you save somewhere information about the sentences deleted by Horus? Information about their owners are especially important because some users use only the sentences owned by particular contributors. In the same way, information about tags and the users who added them is also important. I'm sure we're going to take advantage of this information in the future (when more than one users can add an OK tag, for example).

I also found out that logs and comments sometimes look chaotic when a sentence had been changed.
There are at least three mysteries on this page: http://dev.tatoeba.org/eng/sentences/show/2144030.

1. It looks as if Eckhardtgabriel and yayoi linked this sentence to others when the sentence didn't exist yet, and then CK added the sentence. (This is because the logs don't tell us that Eckhardtgabriel added #483087 "Why you don't take off your coat?")

2. It looks as if CK unlinked #2144030 "Why don't you take off your coat?" from #2143848 "どうしてコートを脱がないのですか." even though they match. (He actually unlinked #483087 "Why you don't take off your coat?" from #2143848.)

3. It looks as if raggione told CK that the word order was wrong and Eckhardtgabriel corrected CK's sentence, which was already correct.

Things would get more complicated when both sentences were changed. We can see the authentic logs of deleted sentences, but we cannot see what really happened before the merger to the ones that survived.

I wish I could make constructive suggestion, but nothing comes to mind now. I'll post again when I have a concrete idea.


By the way, I think there should be a way to go from #2144030 to #483087.
I like loolmeh's idea here https://github.com/Tatoeba/tato...ment-64121681. You can add "#483087を併合" to the logs of #2144030 and "#2144030に併合" to the logs of #483087. If you do this, Horus wouldn't have to post (English) comments anymore.