Wall - Tatoeba

I think it has already been pointed out but maybbe it's a good idea to point it again

The vocabulary feature is a great tool. It helps us to study languages by adding words in languages we don't speak fluently and want to see it used in contexts.

However, looking for https://tatoeba.org/eng/Vocabul...sentences/por, I've found words that don't exist in Portuguese like "decoy", misspelled words like "libro de cabeceira" (it should be "liVro") and also things like "Posso colocar aqui verbos usamos em nossos dia." (I can add here verbs we use every day), "Caso precisem de alguma informação com relação ao Português Brasil posso ajuda-los." (In case you need any information about Brazilian Portuguese I can help you)

http://prntscr.com/nrnhj9

It'd be good if we can correct these requests or even delete some.

https://tatoeba.org/eng/Vocabul..._sentences/eng (English)

Make it up - Помириться (Russian)
communicative language teaching (not a word)
That's what this is all about. - Это то из-за чего это всё. (Russian again)
щзхлзщхлщзх (Can't belive this is in Russian or any other language)

http://prntscr.com/nrnj8o

hide replies

I agree with the problem of the list being cluttered with typos and other mistakes.

I do think that for example "communicative language teaching" is completely fine on the list, though. One might very well understand all the component words and have example sentences with them without understanding the whole term or having it in any sentence.

With English this is especially bad, since the language uses few compound words; "language teaching" rather than "languageteaching", in marked contrast with many Germanic languages and Finnish, for example.

At least in scientific English there are many terms that refer to a concrete thing, but due to features of the language, they are written as a collection of separate words. "magnetic resonance imaging", for example, or "inverse problem".

There are also cases like the Danish "rejse" (to travel) versus "rejse sig" (reflexive form, to get up e.g. from a sitting position). It should be completely valid to have "rejse sig" as a vocabulary item.

Full supprt - I'd like to get my hands at the requests for German, which need to be sorted badly.

hide replies

TRANG,

Would it be possible to export the "vocabulary" data?
If not weekly, at least one time, so we could browse it offline.

language + TAB + vocabulary word(s) + TAB + the username of the person who submitted the word.

Possibly adding one more field with the date added might be interesting, but not absolutely necessary.

Related issue on GitHub: https://github.com/Tatoeba/tatoeba2/issues/1473

To solve this, one possible approach is the one suggested in the GitHub issue: let corpus maintainers edit/delete any vocabulary items. This approach comes with a couple of problems however.

1) What if corpus maintainers edit/delete items that could have been valid?

This problem already shows up here in this thread, as you, Ricardo, said you would have deleted "communicative language teaching", while Thanuir would have kept it.

2) Users don't necessarily add vocabulary to get example sentences.

I can imagine that some users just want to make a list of words/phrases they want to learn and they find it useful to have the translations next to each word/phrase. That's probably why you have those items that contain both English and Russian words. Since the vocabulary feature doesn't allow to connect vocabulary between two languages, users will of course put the two languages into one item if they need to. They're not exactly doing something wrong, they're just adapting the feature to their needs. I feel that deleting or editing their items would not be the proper thing to do.

My suggestion would be to allow contributors to ignore items on the "Sentences wanted" page.
- Ignored items would no longer be displayed on the "Sentences wanted" page.
- Ignored items would be individual, which means that if I ignore an item, it is ignored only for me. If other people don't want to see an item, they would have to ignore it themselves.

In parallel to that, we could split the vocabulary items into two categories:
- A "regular" list where users just create a vocabulary list for themselves.
- A "needs sentences" list where users add vocabulary for which that they explicitly want example sentences. Only vocabulary items in this list would appear on the "Sentences wanted".

Those are my personal ideas. There is no clear decision yet on how we want to handle the "bad" vocabulary items and if you have better ideas, please do share.

hide replies

To me, the root of the problem is that members are unable to put the correct information in the "add vocabulary items" form. And no one can blame them for that. The process of turning an "I want sentences like that" idea into correct values for "language" + "searchable vocabulary item" is indeed difficult, and probably unclear at first. Members don’t exactly know that their vocabulary items are going to be interpreted as search queries, with the number of results displayed, all that listed on a public page. I can see some members are adding whole sentence pairs as vocabulary items, or a semicolon-separated list of items. I can imagine some members assume that what’s under 'my vocabulary items' is only visible to them. I guess some members do not care that much about the flag because they know the language already. After all, when writing down your own vocabulary list, does anyone bother writing the language name?

That is quite of a UX challenge in my opinion. I have no great idea to solve it but there is certainly room for improvement. I’m pretty sure most people do not read the bottom-right "Tips" block that says "Add vocabulary that you are learning. If your vocabulary does not exist yet in Tatoeba, other contributors can add sentences for it."

I don’t think a per-member black list would help. I can make out two categories of items people would like to hide: (1) items that are not meant to achieve 10 sentences (because they are garbage, completely invalid or personal) and (2) items that won’t achieve 10 sentences because of incorrect values. As for category 1, everyone wants to hide them from the public list. As for category 2, they are valid requests that anyone can be interested in, it’s just that Tatoeba isn’t smart enough to show the correct number of sentences.

Splitting the vocabulary items into personal vs. wanted sentences is an interesting idea, but in the end I don’t think it will solve the problem of having "forever 0 sentences" items cluttering the list.

Some other ideas:
How about allowing members to edit the language of their own items?
How about allowing corpus maintainers to edit the language of other members’ items?
How about somehow dividing the vocabulary items into two categories: the ones that can be readily used as search queries, and the ones that cannot. For the ones that cannot, instead of having a link to a search, we allow to directly add sentences that are assumed to match the item and are counted as such.

[this rant needs to be prefaced with my thanks and understanding of the difficulty in designing and maintaining a software project of this complexity on a part time basis]

I've just gone through a random 20 or so sentences and found 2-3 errors that have been highlighted in the comments in the past (both in terms of easily verifiable spelling errors, and also links between non-direct translations). Sometimes the comments are 6 or more years old.
I feel like this site is far too conservative in not allowing people to fix up other people's mistakes or omissions.
I understand that there is a danger of a well meaning but uninformed newcomer from vandalizing a load of content, but it's very disheartening to not be able to fix obvious things.
Also I don't think the concept of links between sentences is all that mysterious, and some more power should be given to normal contributors in this regard.
I.e. as a new member, it's possible for me to link two sentences together by adding a translation which matches an existing sentence, so why should I not also be trusted to unlink sentences from each other?

[end rant]

hide replies

There is a similar, to me surprisingly strong, take on authorship at some other communal websites. Maybe most people are not willing to have others edit their contributions with impunity? There clearly needs to be some restrictions to stop vandalism, in any case.

Getting the right to add and remove links can happen fast. Just translate or add sentences to your strongest language for a while and then ask for the privileges: https://en.wiki.tatoeba.org/art...d-contributors

With smaller languages it might be that there is no highly active user with privileges, which will slow everything down. With bigger languages, the number of active users might be too small. You can at least comment and maybe mark sentences as OK / unsure / not OK. If you later acquire privileges, then you can go through your commented sentences or the sentences you have marked in a particular way, or sentences in a particular language marked in a particular way, and do the clean-up.

If you add a sentence identical to another, those two will be merged and all translations of both sentences will remain as translations.

hide replies

> Just translate or add sentences to your strongest language for a while and then ask for the privileges

My strongest language is English and I'm noticing errors in a secondary language, so I don't think that is achievable; what I'm looking for is 'gardening' functionality.

> maybe mark sentences as OK / unsure / not OK
Yes, this is a great function, but there is there any feedback when someone resolves your 'OK/Not OK' (if ever)?
I don't think there similar functionality to draw attention to a link/relationship.

Also, adding tags should be opened up for any member, no?

hide replies

You can always leave a comment on a sentence to point out problems with it, even if the sentence is not in your strongest language.

If you mark a sentence "OK", "Unsure", or "Not OK", and then the sentence is modified, the marking will then be displayed with "(outdated)" or a similar string. You can always change or remove your marking later, though you won't get a notification reminding you to do it.

The tag system is open to abuse. Requiring people to stick around the site for a while and explicitly ask for the next level of privilege before they can add tags tends to reduce the amount of abuse.

> My strongest language is English and I'm noticing errors in a secondary language, so I don't think that is achievable; what I'm looking for is 'gardening' functionality.

You can get the gardening functionality by being an active contributor for a while. Hence the suggestion on what is the easiest way to do that.

This search, for example, should have pretty much every Irish sentence without a direct translation to English: https://tatoeba.org/eng/sentenc...o=&sort=random

Maybe you could translate those?

Another fruitful thing would be to add sentences in English that others are unlikely to add. Maybe sentences particular to the Irish culture or geography, or maybe sentences related to your expertise (your particular hobbies, work, education, etc.). Another good way of contributing is to check vocabulary people want to see more of, https://tatoeba.org/eng/Vocabul..._sentences/eng , and add sentences that use those words.

People are typically discouraged from contributing in non-native languages, but contributions in endangered (or contructed or dead) languages are less frowned upon or happily accepted. I do not know enough about the situation of Irish to comment further on this.

...

The link https://tatoeba.org/eng/collect...ghanM/outdated should contain your outdated ratings. I think a rating becomes outdated if the sentence is edited.

...

The tags were, at least originally, meant to contain objective information about the sentence; see https://blog.tatoeba.org/2010/1...uidelines.html . However, they might not be curated effectively at the moment.

Lists are a way of collecting arbitrary sentences in an arbitrary connection. I have not used them much; maybe someone else can tell you more.

hide replies

> should have pretty much every Irish sentence without a direct translation to English

Thanks! Have worked through a few of those.

hide replies

Go n-éirí an bóthar leat, a hEoghan.
(Happy Trails, Mr. Eoghan)

> Lists are a way of collecting arbitrary sentences in an arbitrary connection. I have not used them much; maybe someone else can tell you more.

For example, lists can be used to:
- import sentences into a flashcard program like Anki
- collect sentences that you would like translated into another language
- collect sentences in a language that you would like Tatoeba to support
- collect sentences that you want to process on your own computer

One thing that makes lists useful is that you can download them (though currently this is restricted to lists containing 100 or fewer sentences).

> Maybe most people are not willing to have others edit their contributions with impunity?

As they should be. "Obvious mistakes" is a very dangerous concept. Some people might consider something they've never heard in their life an obvious mistake even before asking the author. For inactive users, that's a little bit different, but as an active user, I would be quite upset if someone were to correct my sentences without noticing me.

> it's very disheartening to not be able to fix obvious things.
That's a beginner feeling, trying to do everything by themselves before thinking about the community behind.
If I have a problem with an English sentence, I can contact one of many people.
If I have a problem that need an admin privilege (privilege that I don't have), I contact an admin.
etc. And there's no need to worry about the 30 seconds you would save by doing it by yourself ^^

hide replies

I think that overeager users would cause some damage, but the overall effect on quality would be good.

However, the problem would be malicious users. What if someone created, say, a bot that would replace words in a language with an obscure script with "Pepe the Frog" written in that script? It might take quite a long time before anyone noticed anything. Or started making obscure mistakes or removing or adding single letters to words. Etc. There would have to be a highly convenient way of reverting such edits en masse over vetting them.

I think human oversight in giving people editing privileges is necessary, given the highly fragmented community and large corpus.

...

However, the real problem here is that there are many housekeeping (or gardening) tasks that are not done reliably. This is doubtless true of small languages that do not have sufficiently active native contributors, but also might be the case with English due to the large volume, I guess.

Leaving comments on sentences is not satisfactory, because they might very well be missed. Tags such as @change and NNC are better, since they are task queues. But I meet Norwegian bokmål sentences with @NNC all the time, because there is at least one prolific none-native contributor, but no sufficiently active native ones, at least with privileges. There are lots of unadopted English sentences. Some tags are not objective designators of facts. The same tag can exist in multiple versions.

Is it possible to view all sentences, starting with the ones with the most "not OK" tags? This would also work as a nice queue of sentences requiring actions, and would be something even a newcomer could contribute to by marking sentences.

>> > it's very disheartening to not be able to fix obvious things.

> That's a beginner feeling, trying to do everything by themselves before thinking about the community behind.

@Aiji I think you missed the point I was trying to make in that there were people who had pointed out the problem with the sentence 6+ years ago in the comments and nothing was done.
So the disheartening part includes the aspect that there's no point in commenting either.

hide replies

The fact that someone pointed out a problem years ago and got no response does not mean that it's useless to post a comment now. Something might have changed in between. For instance, perhaps there was no corpus maintainer for the language then, but there is now.

If you post a new comment, it will show up at the top of the most recent comments. Also, if you can figure out the name of a corpus maintainer for the language, you could try adding their name to the comment, preceded by an at-sign (@). They will then get a notification.

I’ve been playing around with our default search ranking algorithm. I insist on the "default" part because that’s what the vast majority of visitors use. I also focus on searches that do not use double quotes or any special trick. Just plain words. Again because that’s what the vast majority of visitors use.

Our current way of ranking results is pretty basic: it searches for sentences that include all the words (eventually stemmed) and sort them by total number of words in the sentence.

A problem with this approach is that the order of the words is ignored. The top result of searching for "you go there" is "There you go!" because it’s a shorter sentence than "You may go there."

Ignoring word order is especially catastrophic on languages without word boundaries, like Chinese, because the searched characters are randomly reordered into something totally unrelated. For example, the results for "可不可" in Chinese are cluttered by irrelevant "不可something". Same for kana words in Japanese.

In order to address this problem, I tentatively tweaked the default ranking algorithm on https://dev.tatoeba.org/ into something that prioritize, in the following order:

1. sentences that contains an exact match (like if searching for ="you go there")
2. sentences having the "longest common subsequence" (LCS, [1])
3. sentences having the least number of words

[1] https://docs.manticoresearch.co...anking-factors

However, I don’t know if this new ranking suits everyone out there. What do you think?

You can compare the search results on https://tatoeba.org/ (old ranking) and https://dev.tatoeba.org/ (new ranking). You can run a search on tatoeba.org, and then add "dev." in the URL bar and press alt+return to open a new tab.

hide replies

I do prefer a ranking that favors exact matches over stemmed matches. Longest common subsequence also sounds good. But sentences having the least number of words are often not the ones I want to see most. I prefer slightly longer ones that give me more context. For that reason, I always choose random ordering. It doesn't always put the sentences that I want at the very top, but at least I have a good chance of finding them without having to go through pages and pages of very short sentences. Also, providing a mix of sentences that is random with regard to sentence length lets people see more diversity. I think that's a good thing.

hide replies

Maybe you wouldn't want this for the default search, but I wonder if it would be possible to add a "minimum word" option to the advanced search. This might prove useful. For example, members could still opt to sort by length, but start by showing sentences that are over a certain length up to 1,000 results.

More involved, perhaps, but an additional idea might be to have a "maximum length" option, too. This would allow members to have search results displayed randomly between 5 and 12 words in length, for example.

hide replies

Yes, I can imagine that a "minimum length" and a "maximum length" option would be useful. However, the nice thing about favoring sentences that meet a certain criterion rather than limiting them to that criterion is that if there are not enough sentences that meet it, you will automatically see the other ones without having to remove the criterion and do another search. I imagine that if I set a minimum and/or maximum length, I would often eliminate some of the fallback sentences I'd like to see, and then I'd have to do a follow-up search.

Indeed, I wouldn't want the default search to be optimized for my particular needs, which would be something like "favor sentences from five to ten words in length". However, I worry that optimizing it for choosing the shortest sentences would pessimize it for people like me, whereas leaving out a criterion of length would allow people to see a variety of sentences, short and long.

If I could choose and there were no computational costs:

1. An exact match of the sentence.
2. Sentences with exact match of the query as part of it.
3. Sentences with all the exact words, but possibly in different order or with other words between them.
4. Sentences with all the words, but with stemming and the order might be different etc.
5. Sentences with all but one of the words (with stemming and could be in any order).
6. Sentences with all but two of the words (with stemming and could be in any order).
7. And so on.
8. Sentences with even a single searched word (with stemming).

Random order within the categories. (Some of the categories could be sorted into even finer subcategories, but probably not worth it.)

For example search: haluan kalastaa tänään
1. Haluan kalastaa tänään.
2. Minä haluan kalastaa tänään, niin kuin eilenkin.
3. Tänään minä haluan kalastaa. Haluan kuitenkin kalastaa tänään.
4. Haluankin kalastaa tänään. Haluatteko te tänään tai huomenna kalastamaan?
5. Haluatteko elokuviin tänään?
6-8. Karhut kalastavat lohia.

The idea would be to first have the precise phrase and then to have increasingly distantly related phrases, which hopefully would still give some understanding of the involved words.

hide replies

@Thanuir @AlanF_US @CK

Thank you for your feedback. I agree about the relative uselessness of having very short sentences showed first. The idea of randomizing the results within a category, like Thanuir said, is appealing (giving the order is deterministic), but I’m afraid it could be a little bit confusing. I temporarily set up https://dev.tatoeba.org/ like that, please let me know what you think.

Or, if we are to rank using the number of words, what would be the ideal number? Not too long and not too short. It depends of the language of course. Here are some stats about the average number of words per sentence in every language on Tatoeba: https://gist.github.com/jiru/81...5917dc18325fc2

I wonder if we could use these numbers to boost the ranking of sentences having a number of words close to the average, with a formula like rank = –abs(average – words)

hide replies

> Or, if we are to rank using the number of words, what would be the ideal number?

When I look at the lists of sentences that I've compiled for my own learning, the length of the sentences does tend to be pretty close to the average for the language (5+ for both Hebrew and Russian).

> Or, if we are to rank using the number of words, what would be the ideal number?

I'm just brainstorming here.

Perhaps starting with slightly less than average length would be a good idea, followed by sentences lengths incrementally up to slightly over the average length, and then possibly followed by a random selection, or followed by alternating shorter and longer sentences.

For example, if the average length of a sentence is 8, something like the following sequence.

6 words
7 words
8 words
9 words
10 words

5 words
11 words

4 words
12 words

3 words
13 words

etc.

One issue with randomness is that reproducing problems or strange behaviour would be more difficult. Maybe displaying the sentences newest first would be an alternative that adds some amount of randomness while retaining reproducability?

...

Using the average number of words, as suggested by @gillux, would create an alternating pattern that @CK suggests, but without the initial emphasis on slightly longer sentences. It would not be terribly difficult to create a function that would have the type of behaviour that @CK wants when the absolute value in gillux's formula was replaced by it. The function would have to be a piecewise defined function with three linear pieces, or a more complicated one. I do not know how computationally expensive it is to deal with a piecewise defined function.

...

One thing I belive would be good would be to show sentences that contain most of the right words, but not all. For example, I tried searching for the English idiom "to talk through one's hat" using two queries: "talk through one's hat" and "He is talking through his hat." with no results. Presumably the sentences are not there. But if the sentence with "Tom" rather than "he" was there, or the sentences with she/her was there, I would not find it. Or the sentence with you or I.

The use case for searching for the idiom might be that I am trying to understand it or that I would like to translate it. Both would be helped by the search finding sentences which match only some of the words in the search query, but presenting them after the sentences with all the words.

hide replies

Yes, we definitely need randomness to be reproducible (and unpredictable, to avoid "rank boost threats") if this is the direction we’re taking. If I give you a search result URL, I expect that you see the same results as me, and that it stays more or less the same for a little time. I believe it is technically possible to produce a random, deterministic and unpredictable order.

I am concerned that boosting sentences having a number of words close to average is going to be detrimental to diversity, because it’s a incentive for contributors to produce standard-sized sentences. Isn’t there a risk of uniformization of the corpus? Or do we actually want more example sentences that are "efficient" and "standard"? I’d like to know @TRANG's opinion on that matter.

The idea of showing newest sentences first (after sorting by exact matches and LCS) is interesting. It surely adds some randomness, but it’s also an incentive to produce new sentences, and it gives more exposure to new or active users.

Since there is no consensus on an alternative to sorting by number of words, for the moment I’m going to change the default search ranking the way I described on the first post of this thread. We may further improve it later on.

hide replies

There is a related idea here.
https://github.com/Tatoeba/tatoeba2/issues/1713

Here's just another brainstorming idea that you might want to consider.

Right now we have these 3 options in the advanced search. (5 with the "reverse" option.)

Sort: shortest newest random

Perhaps you could add more than one of your search sequences to these options.
Perhaps you could come up with one-word descriptions for them, but add a little more detail in the wiki, or even on the advanced search page. Or, just number them

Algorithm 1
First sorted by the number of words and then ...
1. sentences that contains an exact match (like if searching for ="you go there")
2. sentences having the "longest common subsequence" (LCS, [1])
3. sentences having the least number of words

Algorithm 2
Randomly sorted and then ...
1. sentences that contains an exact match (like if searching for ="you go there")
2. sentences having the "longest common subsequence" (LCS, [1])
3. sentences having the least number of words

Algorithm 3
(Something like what Thanuir described.)

Algorithm 4
(Something like what CK described on https://github.com/Tatoeba/tatoeba2/issues/1713)

Perhaps this would be too time-consuming to do, but I think with descriptions somewhere it wouldn't be too confusing.

I did not completely follow the conversation, but I'd like to answer the following

> I am concerned that boosting sentences having a number of words close to average is going to be detrimental to diversity, because it’s a incentive for contributors to produce standard-sized sentences. Isn’t there a risk of uniformization of the corpus? Or do we actually want more example sentences that are "efficient" and "standard"? I’d like to know @TRANG's opinion on that matter.

From my personal experience, that would definitely uniformize the corpus in its shape (not necessarily in its content). As you said, that's not so difficult to imagine that sentences with a number of words close to the average of a language would produce a huge amount of "standard-size" contributions.
For Latin languages for example, we would probably have something close to
Subject + Verb + Article + Adjective + Complement
and still from a personal point of view, that would provide SO many similar and boring sentences to translate, I'm pretty convince that would hinder my contributions. The most interesting sentences are CLEARLY NOT the one around the average. Well of course, you may encounter some nice expressions, or interesting words, but the vast majority would be "I borrowed a pen to Tom." instead of more elaborated, interesting sentences.
I'm always translating from the English corpus, and when I do a lot of translating at the same time, I always end by skipping several sentences because I feel like "AGAIN this sentence ?!" I can only imagine my feeling if the search would be biased to serve me more similar sentences... (I know English is special, but I guess the problem would be similar for the TOP 10 language at least).

hide replies

I'm pretty sure what Gillux is talking about is the "default" simple search at the top of the page and not the advanced search.

Regular members will always be able to use the advanced search to get a lot of variety.

I, personally, seldom use the search at the top each page unless I'm showing students how to use the website.

I prefer using the advanced search, and even more often use my own "dashboard" of preset advanced searches, so I don't have to keep entering the same information on the advanced search page. Anyone is welcome to use this dashboard http://goo.gl/RzP8hV if they like.

> I am concerned that boosting sentences having a number of words close to
> average is going to be detrimental to diversity, because it’s a incentive
> for contributors to produce standard-sized sentences.

The main factor for a diverse corpus is to have a diverse group of contributors, in my opinion. Next to that, the search ranking probably has very little impact on the kind of sentences that people create.

If contributors were paid every time their sentences are displayed in a search result, then I guess that would be a high enough incentive to produce sentences based on the ranking. But even then, unless they earn a living out of it, I think they will still naturally produce standard-sized sentences no matter the ranking because it's just easier to produce such sentences.

So no worries about influencing diversity here.

What we have to consider is: what is the default usage of the search that we're trying to cover?

My personal usage:
- I'm trying to figure out how to say something in a foreign language and I'm missing vocabulary or grammar knowledge.
- I saw a new word/phrase in a foreign language and I want to understand its meaning or see examples of how it's used.

For these use cases, shorter sentences are in general easier to analyze. So it makes sense to order by number of words.
But if the sentence is too short, it may be lacking context and may not be as useful as a longer sentence. So prioritizing average-sized sentences could make sense.
But average-sized sentences might not always be the most useful for everyone either. Randomizing the results also makes sense: it simply means we don't want to make assumptions about what size is "best".

Random order sounds appealing actually, but I wouldn't change to that until we gather more specific information about the issues of ordering by number of words.

I looked at the pageviews for the search in Google Analytics for the month of April.
- Pageviews with order=words: 12,990
- Pageviews with order=random: 8,339
- Pageviews with order=created: 1,198
- Pageviews with order=modified: 259
(Total pageviews for /sentences/search: 223,174)

It seems that when given the choice, people choose in majority to order by words.

hide replies

Thank you for the numbers, that’s valuable information.

This shows that 90% of the visitors making a search are using the "simple search" (top bar or front page), and 10% the advanced search (advanced search page or "more search criteria" block).

> It seems that when given the choice, people choose in majority to order by words.

However it’s not a fair choice because you can’t tell the visitors who made a choice (clicking on the dropdown, examining the choices and choosing) from the ones who didn’t (glancing over the dropdown or not even seeing it, and using the default value). order=words being the default, I believe it is overrepresented.

I find the number for order=random surprisingly high.

hide replies

I agree.

personally, I use order=random in the advanced search because generally I see from there more diversity (whether I use it for translating sentences or for tagging existing ones), since, in a general way, sorting by word order might present similar patterns (which I use whenever I want to translate or tag the same pattern)

hide replies

Same here.

I use the "fewer words first" mode when I use tatoeba to actually find a way to translate something, and the "random" mode when searching for sentences to translate.

If I need exact matches, I still use "fewer words first", but also the syntax like ="gesundheit"

Reproducible randomness is certainly possible from a mathematical/cryptographic point of view. You do need to think carefully about whether it's possible to reverse-engineer personal information from generated pseudo-random (permalink) seeds, though.

Note that the "advanced search" on dev.tatoeba.org doesn't work properly now.

This search should show English sentences with the word "winter" sorted by fewest words.

> https://dev.tatoeba.org/eng/sen...io=&sort=words

hide replies

Thanks. I reverted it back to normal.

Top 20 Languages on Tatoeba.org by Average Word Length

lao 24.315
khm 20.749
bod 20.607
enm 18.888
jpn 18.369
tha 17.711
lzh 14.477
frm 12.074
sot 12.000
crs 12.000
wuu 11.567
yue 11.126
cmn 11.006
non 11.000
roh 10.619
ain 10.608
tsn 10.000
ang 9.583
toki 9.280
iba 9.071

Based on Gillux's data. https://gist.github.com/jiru/81...5917dc18325fc2

Dear Tatoebians,

Recently I have been working on improving the export capabilities of Tatoeba. I created the necessary code base to provide customized exports (as opposed to our generic exports of the Downloads page). In the future, I plan to allow various kind of exports, like all sentences of a specific language, of a specific user, having a specific license, sentence pairs… But for now, I just started with implementing list exports.

You can check this out on our development website https://dev.tatoeba.org/. Log in with the same credentials as here, and then go to the page https://dev.tatoeba.org/exports/index. From there, you should be able to export and download any list you have access to, no matter how big it is.

Feedback is very welcome.

hide replies

I tried with some lists, including one containing almost 400 sentences, and it worked properly.

In the meanwhile, I have a few suggestions:

1) I'd like to see the ID of sentences. For example: I download some lists for whatever reason, and, if I notice any mistake in a sentence and I want to report the mistake on Tatoeba, I would personally find it quicker to copy and paste the ID rather than the sentence.

2) I would find a delete button useful. If I download something and then I notice it isn't the thing I wanted to actually download, I'd like to have the opportunity to delete it, since this would avoid any confusion to the user if he/she has the need to download many things, and, not less important, doing this would free some space on the server.

3) What about mentioning the license of the sentences contained in lists? Maybe this would be helpful if someone wants to reuse them.

I tried downloading some lists containing more than 1,000 sentences and it worked fine. The only drawback is that exported lists are monolingual.

That 100-sentence limitation was a problem. Thanks for working on it.

I just saw that the download button is truncated when the list name is long: http://i63.tinypic.com/15s3cih.png

I would like to see the owners name as a 4th field added to the export.

BTW, it took less than 3 minutes to generate the file for List 907 (over 714,000 sentences) and get it downloaded

I want to know your personal profiles.
What kind of people are you?
Who are you?
Why do you spend so much time om tatoeba?
I'm sure you're nice people. But why not tell more about yourself?

hide replies

Those Tatoeba contributors who choose to say something about themselves will often do so in their individual profile pages. For example, my profile can be seen at <https://tatoeba.org/epo/user/profile/Objectivesea>

However, when I viewed your profile, Fructo, it says nothing about what kind of person you are, who you are or why you spend time on Tatoeba. I'm sure you could modify your profile to set an example for the rest of us as to the sorts of information you think these profiles should contain. Currently, all we are able to learn about you is that you are supposedly six years old, which sounds implausible on its face.

hide replies

There's no your profile. You're posting dead links.

Also, the birthdate on my profile is fake, I was lazy to indicate the real one.

hide replies

There is a typo in the link; just remove the final ">" or click on Objectivesea's user name next to the profile picture.

Also, if you are too lazy to indicate your real birth date, why would you expect others to answer your questions? I'm sure you'll get to know the users here if you just participate actively. Like in other volunteer efforts, you get goodwill by contributing.

hide replies

How about your goodwill? You don't look having some good will placing all random symbols everywhere where they're not appropriate.
I hope you'll learn some better spelling.
After that, you can talk to other people.

Hello. I still need an explanation, please.
What are the useful outcomes of this project?
How anyone could use this project to draw something useful from it?
What are useful applications of it?
Thank you.

hide replies

Various projects use the data we create here.

http://bit.ly/tatoebalinks

hide replies

I put the typing practice game back online at http://type.braul.io .

hide replies

This is great! Could you also update the list of languages to what Tatoeba supports today?

hide replies

Done!

A lot of sentences were not imported, though (some things sqlite importer could not interpret as valid csv). And thus some languages may have no sentences. I'll take a look at this later.

In addition to what CK mentions, Tatoeba, by itself, works as a multilingual dictionary (though mostly of common words).

It has also been used for at least some research in computational linguistics; this Google scholar search should be a good starting point: https://scholar.google.no/schol...atoeba+corpus"

#7903889
Da gibt's ein Problem, das nicht zum ersten Mal passiert ist, beim Übersetzen. Es geht um Unterschiede beim Namengebrauch.

Wie das obere Beispiel zeigt, verwenden wir Ungarn in dem Fall den ganzen Namen. Wenn ich einen Roman übersetzen würde, wäre's wohl kein Problem, weil ich den Vornamen genauso wie den Namen von einem wüsste.

Beim anderen Beispiel ( #6917885 ) :
Wenn ich einen ungarischen Satz schreibe, kann ich einen nach seinem Vornamen siezen: HU "Tom, kommen Sie bitte her." Es würde wohl auf Deutsch lauten: Herr Kröpelin/Glücks/Schmidt/Jackson, kommen Sie bitte her.

(Welcher ein lustiger Satz sein wollte, konnte deswegen nicht ein lustiger Satz werden.)

Das bräuchte eine Lösung.

hide replies

@maaster,

Úgy gondolom, ha 'bemegyünk' valahova el kell/kellene fogadni a 'hely'
írott és íratlan szabályait, előírásait.

Természetesen lehet javaslatot/javaslatokat tenni, ami esetleg
a 'hely' előnyére válik.

Ez a bejegyzés #7903914 mondattal kapcsolatban születhetett.

>>> Ne tőlem kérdezd! Felraktam a faliújságra is problémát.
Te vagy a főnök! Találj ki rá valamit a többivel. <<<

Szégyellem (mint magyar) ezt a hozzáállásodat, hangnemedet, viselkedésedet.

mraz

hide replies

Én meg úgy gondolom, hogy ne akard megváltoztatni a magyar nyelvben kialakult szokásokat csak azért, mert más nyelvben máshogy vannak a dolgok. De téged soha nem érdekelt, hogy egy magyar mondat magyar mondat-e egyáltalán.
Ez egy olyan dolog, amire megoldást kellene találni, nem csak kérdőjelekkel telenyomni a mondatokat, amit nem értesz vagy csupán nem akarsz megérteni.
Bár nem is volt konkrét, értelmezhető hozzászólásod - szokás szerint; viszontkérdésre nem válaszol, mert nem tudsz, mit mondani rá.
A problémával nem foglalkozol, hanem inkább elmész mellette.

Szerintem meg ez nem oké.

Írott és íratlan szabályok: ha esetleg foglalkoznál a dologgal, rájönnél, hogy erre még nincsen szabály, éppen ezt kellene megoldani, nem csak teljes vállszélességgel támadni.

hide replies

@maaster,
Szégyellem (mint magyar) ezt a hozzáállásodat, hangnemedet, viselkedésedet.

Mraz, megoldást kellene találni egy dologra, nem össze-vissza posztolgatni.
Amit ide most ìrtál, köszönő viszonyban sincsen azzal, amivel kellene, hogy legyen.
Nem viselkedsz egyáltalán vezetőhöz méltóan, magyarként még kevésbé; próbálod lehúzni a magyarokat.
Inkább ezen gondolkozz el!
A kollégáimnak voltak kreatív hozzászólásaik. Üdv!

Legyen Barát Tamás, és akkor van egy kis vicc is benne (...Freund Tom...), alá pedig leírva, hogy mi az oka, hogy családnév is jár mellé. Nem lehet mindent pontosan lefordítani, sőt, valamit le se lehet fordítani, úgy, hogy mindkét nyelven közel ugyanazt jelentse. Itt, mi megpróbálkozunk vele.
Kis Pistának ismered a női változatát magyarban? Angolban viszont biztosan van, ha ilyennel futsz össze, lehet Jane Doe is Kis Pista lesz...

hide replies

Itt egy lista arról, hogy különböző nyelveken milyen neveket szoktak erre alkalmazni, magyar is van közte.

https://hu.wikipedia.org/wiki/John_Doe

hide replies

Érdekes volt végignézni.
Ahogy én tapasztalom, a németben a Mustermann nevet használják leggyakrabban kártyakitöltési mintaként.

We now have over 70,000 Spanish sentences with audio.

https://tatoeba.org/eng/audio/index/spa

95% of the audio files are by arh.

https://tatoeba.org/eng/sentenc.../show/6685/und

hide replies

¡Gracias, arh!

** Stats & Graphs **

Tatoeba Stats, Graphs & Charts have been updated:
https://tatoeba.j-langtools.com/allstats/

* Tatoeba Top 30 Languages Interactive Graphs*

Tatoeba Top 30 Languages Interactive Graphs have been updated:
https://tatoeba.j-langtools.com/igraph/
https://tatoeba.j-langtools.com/igraph/share.html

hide replies

thanks :)

The search bar at the top is now appearing much later as the page loads. I find this frustrating. In the past, you could start typing into the search box as soon as it appeared. Now you have to wait for the page to be completely rendered. (I am on an iPhone using Chrome.)

hide replies

I know that this isn't really the solution you're looking for, but you can create a fast-loading "dashboard" that has the search on the top of the page.

http://study.aitech.ac.jp/tatoe...late/links.php
Create a Dashboard of Customized Links for Tatoeba.org

There are also several pre-set searches to help you find sentences to translate.

Once you create the "dashboard" with the language page your want, bookmark the resulting page and you won't need to create it every time you want it.

You can create more than one dashboard with different pairs of languages, if you like.

In fact it is the rest of the page that loads faster the search bar.

In the past the whole page was remaining blank until everything is rendered. We recently changed it so that we display whatever we can as soon as possible, and the rest appears later.

https://github.com/Tatoeba/tatoeba2/pull/1825

hide replies

"We recently changed it so that we display whatever we can as soon as possible, and the rest appears later."

So why is the form for the advanced search appearing at last? It is added to the page by a script? More generally, what is "the rest"?

From the pure U.I. point of view, It is very unpleasant to see elements of the bottom appear before elements of the top, especially if the element of the top is needed. It gives a feeling of unnecessarily waiting. Although the waiting time would be the same the other way, the feel is different since users tend to understand a page "from top to bottom".

hide replies

> So why is the form for the advanced search appearing at last?

It doesn't appear last for me. And shouldn't appear last for you either... If you can make a video of how it shows up for you, that might help.

> More generally, what is "the rest"?

Basically elements that are rendered with Angular Material[1] will take more time to display. The language dropdown is one of those elements. The buttons as well.

> From the pure U.I. point of view, It is very unpleasant to see elements of the bottom
> appear before elements of the top, especially if the element of the top is needed.

This is something I noted in the pull request[2]. In my case it was more an impression that something was broken for a short time, but I understand the feeling, that there's something disturbing about the delayed rendering.

However, having used the website for a couple of weeks now with this delayed rendering, I personally don't feel disturbed by it anymore. But that's maybe influenced by two things:
1) The search bar is most of the time rendered pretty fast for me. I can notice the delay, but it's really just a fraction of a second.
2) I don't search very often when I'm on Tatoeba. Most of the time I look at the Wall and in such case, I'm actually quite fine that the Wall messages appear first and the search bar is rendered later.

Perhaps a compromise would be to revert to the previous behavior (i.e. all blank until everything is rendered) on specific pages. I would guess the homepage and the search results would be the main candidates.

But I would very much like more feedback on this delayed rendering before we proceed on changing things. Does this annoy anyone else, or did most people get used to it?

hide replies

For me, when loading the "advanced search" page, I see the advanced search first and then the regular search appears last at the top of the page. Sometimes there is hardly any delay, so it's not so noticeable.

https://tatoeba.org/eng/sentences/advanced_search

hide replies

On other pages, for example this one "browse English" https://tatoeba.org/eng/sentenc...all_in/eng/und , I have to wait for the whole page to load before I can ...

... see the search bar.
... click the audio button and listen to an audio file.
... add a rating
... add a sentence to a list.

I’m using Firefox. I do notice a delay between the moment the green empty banner shows and the search fields appear. The delay is between one and two seconds. I think we can work on reducing the delay, I already have some ideas I wrote in a Github issue: https://github.com/Tatoeba/tatoeba2/issues/1891

The issue for me is not so much when the elements are displayed, but (a) how long I have to wait to start typing into the advanced search form (which I use more than the regular search because I prefer random ordering to shortest-first) and (b) whether the elements on the page resize themselves while I'm trying to do an operation (such as scrolling to the top). Before the recent change, I could count on being able to type into the advanced search form (the same way that DostKaplan could count on being able to type into the regular search form) even when it was loading. Now that's no longer the case.

On a similar subject, I generally have multiple tabs (maybe around 10) open in Chrome on an Android tablet or phone, and one of them is generally a Tatoeba advanced search page that I want to reuse in order to perform a new advanced search. When I reactivate it, I find that the page tries to redo the last advanced search, which I no longer care about, before I can perform the new search. This doesn't happen when I use Firefox on Windows. I'm not sure whether this is behavior that is under our control, but if the browser didn't try to redo the old search, I wouldn't need to care as much about the rendering order. Also, searching on a tablet would be faster for me in general. The load on the server might also be reduced, making searches faster for everyone.

I created a faster-loading, cut-down version of the Advanced Search page that you could use that might save you time.

http://bit.ly/tatadv

If you like it, you can bookmark it and jump to it whenever you want to make a search.

There is an explanation at the bottom of that page about it.

hide replies

@CK
It would be more useful if the submit button posts the results to a _named_ window/tab. That way, the results always appear in a separate, same window on submission of the search request. As it is right now, the results appear in the same window/tab as the cut-down version of the Advanced Search page. This means that I can't simply "backspace" once to quickly do another search if I had been hitting "next" multiple times to view the results of my previous search. And doing all this in Chrome on an iPhone means I don't have a bookmark right on the screen that I can access easily (like you can on a desktop or laptop). To get to the bookmark involves several taps. Displaying the results in a named window would help.

Also, what must I choose on the cut-down version of the advanced search page to NOT show results that don't have translations in the target language? Example:

English-->Turkish
Search text: rendezvous
Results: I see English sentences without Turkish translations.

Tips

Latest messages

Wall (5346 threads)

Need some help?

Developers

About