DATAMUNDI

Human * Artificial Intelligence

Our customers are inventing, building and improving self-learning machines. We help our customers to fuel these AI engines, and to measure how intelligent they actually are.

We turn crowd knowhow into datasets NLP researchers can use. We work with our customers to tailor fit the tools we develop for them and that our freelance team is using.

Typically we help to improve sentence segmentation, domain specific terminology, monolingual and bilingual training sets. We create feature sets and we measure progress in accuracy and fluency of the translation output of NMT systems. We build tools that allow humans to verify and fix what happens inside the neural networks producing translations or spoken text.

We work with freelancers from all around the globe, all sharing our passion for:

Language – we go the extra mile for low density languages, and another mile for profanity
Technology – we value human wisdom more than machine intelligence
Quality – we assure our deliveries through human evaluation, meta data and statistics
Data – we create, validate and fix training data and test data

If you have technical skills you think we value, please mail us: recruitment@datamundi.space. We are always looking for developers and web crawl specialists.

Datamundi is a Belgian BV.

A non-exhaustive list of work types

Translated Segment Evaluation

This is a contrastive evaluation where at least 2 freelancers score the same dataset using a Likert scale; In one dataset there are up to 4 contrastive translations per source text. This job is used to measure the quality of competing engines, and to identify added profanity. The Likert scale has 3 scores for a bad translation and 3 scores for a good translation. All scores come with clear definitions that need to be followed strictly by the evaluators.

Contrastive Translation Evaluation

Evaluate the source term and label 2 to 4 contrastive translations. Occasionally we have a non-contrastive evaluation job as well. Translators are also invited to add their own translation variant, if needed.

Collective Dictionary Translation

Many freelancers work together on translating the same dictionary; there is some artificial overlap so we can see how much variation and disagreement there is.

Guaranteed Human Translation

Translate a source string, out of context, from scratch, without using any kind of Machine translation, strictly following detailed instructions from a translation guidelines document.

Binary Translation Evaluation

Two or more freelancers score the same dataset using a binary scale; This job is used to measure the quality of competing engines and to measure the evolution of engines over time. The definition of good is simple and straightforward; for bad, the instructions contain some clear examples.

Parallel Data Collection

One or more freelancers score the quality of parallel web pages, tag terms, flag profanities... We use many different tools for this.

Fails Quality Tagging

Evaluators analyze a translated sentence, identify and classify all errors in the translation, and markup the issues in source (missing tokens) or target (quality fails).

URL Pair Qualification

The evaluators will see 2 comparable pages in a web browser. They classify and qualify the pages.

Token Pair Markup

The evaluators will see 2 parallel pages in a web browser. They markup 5 perfect translations of 5 to 8 words in the source and the target panes.

Profanity Identification Task

The evaluators will browse through monolingual web pages, confirm the language, and then markup vulgar terms, blasphemy, slurs, ...

Language Variant Identification

The evaluators browse through a list sentences and flag them if the sentence can be used for training a language variant engine.

Collective Profanity Acquisition

Submit vulgar, offensive, racist, sexist... language with a team of freelancers, inspired by a list of English profanity.

Human Language Identification

Label strings based on the language you recognize; the strings shown are all strings language identification tools cannot identify correctly.

Human Alignment Annotation

Linguists evaluate the validity of sentence alignments from your automated alignment algorithms. The job is split in 2 steps: scoring target alignment (bag-of-words) and scoring translation quality (binary). Evaluators can review the assessment before submission and they can review all submission on a grid. Profane content can be reported as well.

Translation Quality Tagging

Quality tagging on human translations and machine translations based on the SAE J2450 quality matrix (automotive standard).

Abstract Meaning Representation

Abstract meaning representation involves rendering a sentence and its meaning in the form of a rooted, directed, acyclic graph. The graph is composed of concepts (events, entities) and the relations between these concepts. In more general terms, AMR captures the main event, who is doing what to whom, when, where, etc. Since the focus is on capturing the meaning, AMR strives for a logical, less syntactical representation.

Quality Assurance Job

Used to track the quality of deliveries over time. The result of this is shared with the freelancer if required.

Human Annotation of Websites

Annotate a list of websites using an evaluation form

Landing Page Labeling

Detect the pages in websites that are frequently updated

Monolingual Segmentation Tagging

List examples of sentences explaining the segmentation rule typical to your mother tongue

Bilingual Segmentation Tagging

Map a set of tagged source sentences to the segmentation breakers/non-breakers in your mother tongue

Domain Root Inventory

List websites that match the description of a given (narrow) vertical domain.

Web Crawling and Text Extraction

If you want to join us, this is what you should know

We work with freelancers only. Together we create data that researchers can use. To enrich the customer data, we measure the time our freelancers spend on the job in detail. For some work types we also measure mouse miles and the corrections on given text and created text. This meta data is used:

to compare the effort done by the evaluators;
to verify the predicted value of the job is fair;
to trigger quality control;
to verify how stable the evaluations are over time.

For all jobs, a good to perfect level of written and spoken English is mandatory.

From 2020 all our tools will be done on an online Job Portal, through which our freelancers can generate their invoice. You will need a decent connection to the internet, and a screen with a resolution of 1024 * 800 pixels, and use Google Chrome. You can work on Windows, MacOS or Linux. You cannot work on a mobile phone.

We are always looking for people who have good knowledge of insulting language, sexist & racist terms and idiomatic expressions, slurs…

Professional translators and other language professionals are welcome to join our team. We’ll interview and test you so we know what work types you can handle.

We take pride in fair and fast payments. Our Job Portal facilitates this. We pay via SEPA bank wire payments, PayPal, XOOM, Skrill & Neteller.

If you would like to be part of the international team of language specialists taking paid NLP research jobs, mail us at recruitment@datamundi.space. If we have jobs, we’ll setup some tests first. If you’re only looking for translation jobs, please don’t mail us.