Cognitive aspects of lexicon in the light of the language picture of the world
The words representing these concepts were translated into the two target languages, German and Italian. A statistical analysis (using Tukey’s honestly significant difference test as implemented in the R toolkit 2) of word length distributions (within and across categories) showed no significant differences in either language. There were instead significant differences in the frequency of target words, as collected from the German, Italian and English WaCky corpora3. In particular, words of the class body part had significantly larger frequencies across languages than the words of the other classes (not surprisingly, the words eye, head and hand appear much more often in corpora than the other words in the stimuli list).
Experimental Procedure
The participants in the concept description experiment were students attending the last 3 years of a German or Italian high school and reported to be native speakers of the respective languages. 73 German and 69 Italian students participated in the experiment, with ages ranging between 15 and 19.
The average age was 16.7 (standard deviation 0.92) for Germans and 16.8 (s.d. 0.70) for Italians. The experiment was conducted group-wise in schools. Each participant was provided with a random set of 25 concepts, each presented on a separate sheet of paper. To have an equal number of participants describing each concept, for each randomly matched subject pair the whole set of concepts was randomised and divided into 2 subsets.
Each subject saw the target stimuli in his/her subset in a different random order (due to technical problems, the split was not always different across subject pairs).
Short instructions were provided orally before the experiment, and repeated in written format on the front cover of the questionnaire booklet distributed to each subject. To make the concept description task more natural, we suggested that participants should imagine a group of alien visitors, to each of which a particular word for a concrete object was unknown and thus had to be described.
Participants should assume that each alien visitor knew all other words of the language apart from the unknown (target) word.
Participants were asked to enter a descriptive phrase per line (not necessarily a whole sentence) and to try and write at least 4 phrases per word.
They were given a maximum of one minute per concept, and they were not allowed to go back to the previous pages.
Before the real experiment, subjects were presented an example concept (not in the target list) and were encouraged to describe it while asking clarifications about the task.
All subjects returned the questionnaire so that for a concept we obtained, on average, descriptions by German subjects
Transcription and Normalization
The collected data were digitally transcribed and responses were manually checked to make sure that phrases denoting different properties had been properly split. We tried to systematically apply the criterion that, if at least one participant produced 2 properties on separate lines, then the properties would always be split in the rest of the data set.
However, this approach was not always equally applicable in both languages. For example, Trans-portmittel (German) and mezzo di trasporto (Italian) both are compounds used as hyponyms for what English speakers would probably rather classify as vehicles. In contrast to Transportmittel, mezzo di trasporto is splittable as mezzo, that can also be used on its own to refer to a kind of vehicle (and is defined more specifically by adding the fact that it is used for transportation). The German compound word also refers to the function of transportation, but -mittel has a rather general meaning, and would not be used alone to refer to a vehicle.
Hence, Transportmittel was kept as a whole and the Italian quasi-equivalent was split, possibly creating a bias between the two data sets (if the Italian string is split into mezzo and trasporto, these will be later classified as hypernym and functional features, respectively; if the German word is not split, it will only receive one of these type labels). More in general, note that in German compounds are written as single orthographic words, whereas in Italian the equivalent concepts are often expressed by several words. This could also create further bias in the data annotation and hence in the analysis.
Data were then normalized and transcribed into English, before annotating the type of semantic relation. Normalization was done in accordance with McRae et al. (2005), using their feature norms as guidelines, and it included leaving habitual words like “normally,”, “often”, “most” etc. out, as they just express the typicality of the concept description, which is the implicit task.
Mapping to Relation Types
Normalized and translated phrases were subsequently labeled for relation types following McRae et al.’s criteria and using a subset of the semantic relation types described in Wu and Barsalou (2004): see section 4.1 below for the list of relations used in the current analysis.
Trying to adapt the annotation style to that of McRae et al., we encountered some dubious cases.
For example, in the McRae et al.’s norms, carnivore is classified as a hypernym, but eats meat as a behavior, whereas they seem to us to convey essentially the same information. In this case, we decided to map both to eats meat (behavior).
Among other surprising choices, the normalized phrase used for cargo is seen by McRae et al. as a function, but used by passengers is classified as denoting the participants in a situation. In this case, we followed their policy.
While we tried to be consistent in relation labelling within and across languages, it is likely that our own normalization and type mapping also include a number of inconsistencies, and our results must be interpreted by keeping this important caveat in mind.
The average number of normalized phrases obtained for a concept presented is 5.24 (s.d. 1.82) for the German participants and 4.96 (s.d. 1.86) for the Italian participants; in total, for a concept in our set, the following number of phrases was obtained on average: 191.28 (German, s.d. 25.96) and 170.42 (Italian, s.d. 25.49).
4 Results
The distribution of property types is analyzed both class-independently and within each class (separately for German and Italian), and an unsupervised clustering analysis based on property types is conducted.
Distributional Analysis
We first look at the issue of how comparable the German and Italian data are, starting with a check of the overlap at the level of specific properties.
There are 226 concept–property pairs that were produced by at least 10 German subjects; 260 pairs were produced by at least 10 Italians. Among these common pairs, 156 (i. e., 69% of the total German pairs, and 60% of the Italian pairs) are shared across the 2 languages. This suggests that the two sets are quite similar, since the overlap of specific pairs is strongly affected by small differences in normalization (e. g., has a fur, has fur and is hairy count as completely different properties).
Of greater interest to us is to check to what extent property types vary across languages and across concept classes. In order to focus on the main patterns emerging from the data, we limit our analysis to the 6 most common property types in the whole data set (that are also the top 6 types in the two languages separately), accounting for 69% of the overall responses. These types are:
• (external) part (WB code: ece; “dog has 4 legs”)
• (external) quality (WB code: ese; “apple is green”)
• behaviour (WB code: eb; “dog barks”)
• function (WB code: sf ; “broom is for sweeping”)
• location (WB code: sl; “skyscraper is found in cities”)
Figure 1 compares the distribution of property types in the two languages via a mosaic plot (Meyer et al., 2006), where rectangles have areas proportional to observed frequencies in the corresponding cells. The overall distribution is very similar. The only significant differences pertain to category and location types: Both differences are significant at the level p < 0.0001, according to a Pearson residual test (Zeileis et al., 2005).
For the difference in location, no clear pattern emerges from a qualitative analysis of German and Italian location properties. Regarding the difference in (superordinate) categories, we find, interestingly, a small set of more or less abstract hypernyms that are frequently produced by Italians, but never by Germans: construction (72), object (36), structure (16). In the these cases, the Italian translations have subtle shades of meaning that make them more likely to be used than their German counterparts. For example, the Italian word oggetto (“object”) is used somewhat more concretely than the extremely abstract German word Objekt (or English “object”, for that matter) – in Italian, the word might carry more of an “artifact, man-made item” meaning. At the same time, oggetto is less colloquial than German Sache, and thus more amenable to be entered in a written definition. In addition, among others, the category vehicle was more frequent in the Italian than in the German data set (for which one reason could be the difference between the German and Italian equivalents, which was discussed in section 3.3). Differences of this sort remind us that property elicitation is first and foremost a verbal task, and as such it is constrained by language-specific usages. It is left to future research to test to what extent linguistic constraints also affect deeper conceptual representations (would Italians be faster than Germans type at recognizing super ordinate properties of concepts when they are expressed non-verbally?).
Despite the differences we just discussed, the main trend emerging is one of essential agreement between the two languages, and indicates that, with some caveats, salient property types may be cross-linguistically robust. We, thus, turn to the issue of how such types are distributed across concepts of different classes. This question is visually answered by the association plots on the following page.
Each plot illustrates, through rectangle heights, how much each cell deviates from the value expected given the overall contingency tables (in our case, the reference contingency tables are the language-specific distributions). The sign of the deviation is coded by direction with respect to the baseline. For example, the first row of the left plot tells us, among other things, that in German behavior properties are strongly over-represented in mammals, whereas function properties are under-represented within this class.
The first observation we can make about figure 2 is how, for both languages, a large proportion of cells show a significant departure from the overall distribution. This confirms what has already been observed and reported in the literature on English norms – see, in particular, Vinson et. al. (2008): property types are highly distinctive characteristics of concept classes.
The class-specific distributions are extremely similar in German and Italian. There is no single case in which the same cell is deviating significantly but in opposite directions in the two languages; and the most common pattern by far is the one in which the two languages show the same deviation profile across cells, often with very similar effect sizes (compare, e. g., the behaviour and function columns). These results suggest that property types are not much affected by linguistic factors, an intrinsically interesting finding that also supports our idea of structuring relation-based navigation in a multi-lingual dictionary using concept-class–specific property types.
The type patterns associated with specific concept classes are not particularly surprising, and they have been already observed in previous studies (Vinson and Vigliocco, 2008; Baroni and Lenci, 2008). In particular, living things (animals and plants) are characterized by paucity of functional features, that instead characterise all man-made concepts. Within the living things, animals are characterised by typical behaviours (they bark, fly, etc.) and, to a lesser extent, parts (they have legs, wings, etc.), whereas plants are characterised by a wealth of qualities (they are sweet, yellow, etc.)
Differences are less pronounced within man-made objects, but we can observe parts as typical of tool and furniture descriptions. Finally, location is a more typical definitional characteristic of buildings (for clothing, nothing stands out, if not, perhaps, the pronounced lack of association with typical locations). Body parts, interestingly, have a type profile that is very similar to the one of (manipulable) tools – manipulable objects are, after all, extensions of our bodies.
Clustering by Property Types
The distributional analysis presented in the previous section confirmed our main hypotheses – that property types are salient properties of concepts that differ from a concept class to the other, but are robust across languages. However, we did not take skewing effects associated to specific concepts into account (e. g., it could be that, say, the property profile we observe for body parts in figure 2 is really a deceiving average of completely opposite patterns associated to, say, heads and hands).
Moreover, our analysis already assumed a division into classes – but the type patterns, e. g., of mammals and birds are very similar, suggesting that a higher-level “animal” class would be more appropriate when structuring concepts in terms of type profiles.