-Introduction
-
-12dicts is a collection of English word lists. It differs in several
-important ways from most of the other free word lists you can download.
-
- * The 12dicts lists are oriented towards common words. If you're
- looking for myriads of archaic, scientific or computer jargon words,
- you should look elsewhere.
- * The 12dicts lists have been rigorously checked for errors. (This is
- not to say that they are error-free, merely that enough care has
- been taken that errors are rather infrequent.)
- * 12dicts contains a variety of lists, of different sizes and
- characteristics. One size does not fit all. Because each list has
- different characteristics, I do not recommend combining them, except
- as noted below.
-
-Originally, 12dicts was composed of lists derived from a specific set of
-12 source dictionaries. In addition to these "classic" lists, 12dicts
-now includes lists derived from other sources. It would perhaps be
-appropriate to rename 12dicts to something more generic, such as BAWL
-(Beale's Assorted Word Lists), but I have not done so in order to
-preserve continuity.
-
-A quick summary of the 12dicts lists and their characteristics is as
-follows:
-
-+---------------------------------------------------------+
-| |3esl |6of12|2of12|2of4brif|5desk|2of12inf|
-|---------------+-----+-----+-----+--------+-----+--------|
-|Size |21877|32153|41236|60387 |61406|81520 |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Abbreviations |Y |Y |N |N |N |N |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Acronyms |Y |Y |N |N |Y |N |
-|---------------+-----+-----+-----+--------+-----+--------|
-|British English|N |N |N |Y |N |N |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Hyphenations |Y |Y |Y |N |N |N |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Inflections |N |N |N |Y |N |Y |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Names |Y |Y |N |N |Y |N |
-|---------------+-----+-----+-----+--------+-----+--------|
-|Phrases |Y |Y |N |N |N |N |
-+---------------------------------------------------------+
+Version 6 of the 12dicts word lists
+
+Welcome to version 6 of 12dicts, a collection of English word lists. It differs
+in several important ways from most of the other free word lists you can
+download.
+
+ • The 12dicts lists are oriented towards common words. If you're looking for
+ myriads of archaic, scientific or computer jargon words, you should look
+ elsewhere.
+ • The 12dicts lists have been rigorously checked for errors. (This is not to
+ say that they are error-free, merely that enough care has been taken that
+ errors are rather infrequent.)
+ • 12dicts contains a variety of lists, of different sizes and
+ characteristics. One size does not fit all. Because each list has different
+ characteristics, I do not recommend combining them, except as noted below.
+
+Originally, 12dicts was composed of lists derived from a specific set of 12
+source dictionaries. In addition to these "classic" lists, 12dicts now includes
+lists derived from other sources. It would perhaps be appropriate to rename
+12dicts to something more generic, such as BAWL (Beale's Assorted Word Lists),
+but I have not done so in order to preserve continuity.
The remainder of this document is organized as follows:
- * This release
- * The classic 12dicts lists
- + The 6of12 and 2of12 lists
- + The 2of12inf list
- * The 3esl list
- * The 2of4brif list
- * The 5desk list
- * How 12dicts came to be
- * Conclusions
+ • This release
+ • Some general observations
+ • The organization of 12dicts
+ • Picking a list to use
+ • The classic (American) 12dicts lists
+ □ The 6of12 and 2of12 lists
+ □ The 2of12inf list
+ □ The 3esl list
+ • The international 12dicts lists
+ □ The 2of4brif list
+ □ The 3of6 lists
+ □ The 5d+2a list
+ • The lemmatized 12dicts lists
+ □ The 2+2+3lem list
+ □ The 2+2+3frq list
+ □ The 2+2+3cmn list
+ • Specialized 12dicts lists
+ □ The neol2016 list
+ □ The 2of5core list
+ □ The 6phrase list
+ • How 12dicts came to be
+ • My other projects
+ • Conclusions
This release
-This is release 4.0 of 12dicts, released Jan. 18, 2003. It differs from
-previous versions by containing three additional lists which are not
-derived from the "classic" 12dicts sources. Changes to the classic lists
-are limited to error corrections.
-
-The classic 12dicts lists
-
-The 12dicts project began as the n-dicts projects, n being a variable
-whose value finally stabilized as 12. The purpose of the project was to
-create a list of words approximating the common core of the vocabulary
-of American English.
-
-The methodology of the project was to record and correlate the words
-listed in a number of small dictionaries. The number of dictionaries so
-recorded is now 12, comprising 8 ESL (English as a Second Language)
-dictionaries and 4 "desk dictionaries". The dictionaries chosen vary
-widely by publisher, by style, by completeness and by depth. In this
-version of 12dicts, all of them are dictionaries of American English
-(three from British publishers). The smallest of them contains about
-20,000 entries, and the largest 46,000. (All totaled, there are about
-75,000 entries, many of which appear in only a single dictionary.) All
-but two of them were published in the last seven years.
+This is release 6.0.1 of 12dicts, released April 2016. This is a major release.
+The following is a brief rundown of the changes and additions:
+
+ • A number of new lists, based on 6 "advanced learner's" ESL dictionaries,
+ have been added. The sources are reasonably balanced between American and
+ British English. In addition to 3of6game.txt and 3of6all.txt, which are
+ more or less traditional word lists, 6phrase.txt, a list of multi-word
+ phrases, was added.
+ • The 5desk.txt list has been augmented with words from two of the advanced
+ learner's dictionaries, and renamed 5d+2a.txt to reflect this change.
+ • The lemmatized lists have been augmented by adding words from the new
+ advanced learner list 3of6game.txt along with some commonly-used hyphenated
+ words from both 2of12.txt and 3of6all.txt. These lists have been renamed
+ from 2+2lemma.txt and 2+2gfreq.txt to 2+2+3lem.txt and 2+2+3frq.txt to
+ reflect this change.
+ • Word frequency information for the lemmatized frequency list is now
+ obtained from a BYU corpus-derived frequency list rather than from Google
+ web data. A small number of abbreviations and proper names have been added
+ to the list.
+ • Two new small lists of especially common or important words have been
+ added: 2of5core.txt and 2+2+3cmn.txt.
+ • The annotations of the 6of12.txt list have been reworked.
+ • Minor corrections have been made to the "classic" lists.
+ • The neologism file, containing words too recent or controversial to be
+ listed in many of the source dictionaries, has been updated.
+ • Slight changes were made to the list of 6of12.txt signature words after it
+ was determined that a few of them should have been present as regular
+ (non-signature) words in the main body of the list but were omitted due to
+ compilation errors.
+ • The files were organized into directories to make them more manageable
+ given their increased number.
+ • The 2of4brif.txt list is being "deprecated". I will continue to distribute
+ it, but will not be changing or maintaining it. I consider the 3of6game.txt
+ list to be a complete replacement.
+
+Some general observations
+
+With the exception of the neol2016 list, all the 12dicts lists were assembled
+in a similar fashion. Words were extracted from a set of source dictionaries
+and, in most cases, a list was assembled by selecting all words and phrases
+present in some number of the sources meeting certain criteria. For instance,
+the 2of12 list comprises lower-case and hyphenated words present in at least
+two of twelve source dictionaries. For some lists, rules are added establishing
+exceptions for certain words or classes of words - for instance, the 2of12 list
+contains the upper-case words I and O as exceptions to its general exclusion of
+upper-case words and names.
+
+Some lists contain annotations, which are special characters appended to
+certain words. For instance, the ":" character is used in some lists to
+identify abbreviations which are ordinarily used without a terminating period.
+This annotation allows these abbreviations to be distinguished from possibly
+similar regular words. Another annotation, used in the 3of6game and 3of6all
+lists, is the "$" character, indicating a word that was placed in the list even
+though fewer than three of the sources mention it. The "+" and "!'" annotations
+are used to identify signature words and neologisms, as described below. Note
+that is it possible for a word to have more than one annotation, though this is
+uncommon. For instance, in the 6of12 list, the word boldfaced~= has both a "~"
+and a "=" annotation, signifying that the word was an arbitrary choice between
+two equally attested forms (boldfaced and bold-faced), and that it was not
+given a separate definition in a majority of the sources listing it.
+
+A number of the lists contain signature words. These are words (or phrases)
+which do not meet the formal criteria for inclusion in a list, but which I have
+chosen to add anyway, as words which "ought to be" present. Whether a list
+contains signature words depends on the specific list. Usually, but not always,
+a signature word is present in some of the sources used for a list, but not
+enough of them to qualify for inclusion on that basis. Some lists may "inherit"
+signature words from other lists from which they were assembled. For instance,
+the 6phrase list includes the signature words from the 3of6all list. In most
+cases, signature words are marked with the "+" annotation.
+
+The neol2016 list contains neologisms, words which are not listed in some or
+all of the source dictionaries for 12dicts, generally for one of two reasons.
+First, many of the words are recent coinages which were not yet fully
+recognized by mainstream lexicographers when the 12dicts sources were
+published. Examples of such words are selfie, Obamacare, emoji and snarky.
+Other so-called neologisms are well-established, often well-known, words which
+are considered scandalous, such as sexual slang and ethnic slurs, and which are
+often deliberately omitted from dictionaries. (I will not give any examples of
+this sort of word here, but you will find some in the neol2016 list.) Note that
+the neologism list has been accumulating for about fifteen years now, and some
+of its words have become almost old-fashioned, such as spam and dotcom. The
+neologism list is provided so that some or all of its words can be added to the
+other lists where the intended usage makes that appropriate. However, I have
+added the single-word neologisms to the 2of12inf and 3of6game, as these lists
+are the most likely to be used in coding word games, where it is desirable to
+recognize the very latest hot vocabulary. In these lists, neologisms are
+annotated with the "!" character.
+
+One other observation worth making is about diacritics. Some dictionaries will
+tell you that there are English words correctly spelled café, naïve, façade and
+piñata, and I do not wish to disagree with these authorities. But as a
+practical matter, Americans do not like to use diacritics. Furthermore they use
+keyboards which do not contain accented letters, and are often unfamiliar with
+the often clumsy techniques that their software provides to use such
+characters. For this reason, 12dicts drops all the accents from its English
+vocabulary. This is particularly valuable for coding word games, where
+expecting players to accent the e in cafe is not going to make them happy. (I
+cannot help pointing out that Scrabble® contains no É tiles.) I apologize to
+those who consider it a matter of some emotional importance that resume and
+résumé should be differently spelled.
+
+The organization of 12dicts
+
+The 12dicts lists are organized into four directories, grouping lists with
+similar characteristics together. The remainder of this document follows this
+organization as well. For each directory, a section of the documentation
+describes in detail the lists it contains.
+
+Most users of 12dicts end up using only a single list. If it is clear which
+directory will contain the list you need, you can go directly to the
+appropriate documentation.
+
+The four directories are:
+
+ • American. The lists in this directory contain primarily American English
+ words.
+ • International. The lists in this directory contain words from both American
+ English and British English.
+ • Lemmatized. The lists in this directory combine other lists, and are
+ formatted in a way that clarifies word relationships.
+ • Special. The lists in this directory are special-purpose lists that do not
+ fit into the other directories.
+
+Picking a list to use
+
+If you are not certain which directory might contain the kind of list you are
+looking for, here is a breakdown of the 12dicts lists by size and purpose which
+may be helpful. If it does not help you find what you are looking for, you
+might want to check out this table, which summarizes the characteristics of all
+the 12dicts files, put together by Kevin Atkinson. Also, I suggest reading the
+introduction to each directory presented in the previous paragraph, each of
+which contains a table summarizing exactly what you can expect from each list
+in that directory.
+
+ • Lists for use in word games: 2of12inf (American), 3of6game (International).
+ • A list ordered by word frequency: 2+2+3frq (Lemmatized).
+ • Small lists of common words: 2of5core (Special, very small), 3esl
+ (American), 2+2+3cmn (Lemmatized).
+ • Medium-sized lists: 6of12 (American, smaller, includes phrases), 2of12
+ (American, larger, no phrases).
+ • Large lists: 3of6all (International, includes phrases), 5d+2a
+ (International, no phrases, many obscure words), 2+2+3lem (Lemmatized, very
+ large).
+ • A list of phrases: 6phrase (Special).
+
+The classic (American) 12dicts lists
+
+The 12dicts project began as the n-dicts projects, n being a variable whose
+value finally stabilized as 12. The purpose of the project was to create a list
+of words approximating the common core of the vocabulary of American English.
+
+The methodology of the project was to record and correlate the words listed in
+a number of small dictionaries. The number of dictionaries so recorded ended up
+as 12, comprising 8 ESL (English as a Second Language) dictionaries and 4 "desk
+dictionaries". The dictionaries chosen varied widely by publisher, by style, by
+completeness and by depth. All of them were dictionaries of American English
+(three from British publishers). The smallest of them contained about 20,000
+entries, and the largest 46,000. (All totaled, there are about 75,000 entries,
+many of which appeared in only a single dictionary.) All but two of the sources
+were published between 1992 and 1999, when 12dicts was first released.
+
+The following table summarizes the contents of each of the classic lists,
+located in the American directory, ordered by size in words:
+
+┌─────────────────┬──────┬──────┬──────┬────────┐
+│ │ 3esl │6of12 │2of12 │2of12inf│
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Size (Words) │22,000│32,000│41,000│82,000 │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Number of Sources│3 │12 │12 │12 │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│American English │Y │Y │Y │Y │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│British English │– │– │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Ordinary words │Y │Y │Y │Y │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Inflections │– │– │– │Y │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Hyphenations │Y │Y │Y │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Phrases │Y │Y │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Names │Y │Y │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Abbreviations │Y │Y │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Acronyms │Y │Y │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Prefixes/Suffixes│– │– │– │– │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Signature words │– │Y │Y │* │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Neologisms │– │– │– │Y │
+├─────────────────┼──────┼──────┼──────┼────────┤
+│Annotations │Y │Y │Y │Y │
+└─────────────────┴──────┴──────┴──────┴────────┘
+
+A * in the "Signature Words" row means that signature words associated with
+some other list may be present, but there are no signature words associated
+specifically with that list.
The 6of12 and 2of12 lists
-I initially tried two different ways of winnowing the 12dicts data to
-produce lists of common words. Both produced interesting results. One
-list, the 6of12 list, contains all words and phrases listed in 6 of the
-12 dictionaries. One way of describing this list is that it contains
-those words and phrases which a (seeming) majority of lexicographers
-believe are relevant to people learning English, and/or to everyday
-usage. This list contains about 32,000 words and phrases. The other
-list, the 2of12 list, is more inclusive in that it includes words listed
-in as few as two of the source dictionaries, but less inclusive in that
-it excludes items of various sorts, including multiword phrases, proper
-names and abbreviations. This list contains about 41,000 words. It is
-perhaps more suitable for use in areas like spell checking or word games
-than the 6of12 list. (Honesty compels me to admit that neither of these
-lists is, by itself, a good choice for spell checking, due to the
-absence of inflections, proper names, Roman numerals, etc.)
-
-A third list, 2of12inf.txt, developed later, is of a rather different
+I initially tried two different ways of winnowing the 12dicts data to produce
+lists of common words. Both produced interesting results. One list, the 6of12
+list, contained all words and phrases listed in 6 of the 12 dictionaries. One
+way of describing this list is that it contains those words and phrases which a
+(seeming) majority of lexicographers believe are relevant to people learning
+English, and/or to everyday usage. This list contained about 32,000 words and
+phrases. The other list, the 2of12 list, was more inclusive in that it included
+words listed in as few as two of the source dictionaries, but less inclusive in
+that it excluded items of various sorts, including multi-word phrases, proper
+names and abbreviations. This list contained about 41,000 words. It was likely
+more suitable for use in areas like spell checking or word games than the 6of12
+list. (Honesty compels me to admit that neither of these lists is, by itself, a
+good choice for spell checking, due to the absence of inflections, proper
+names, Roman numerals, etc.)
+
+A third list, 2of12inf.txt, developed later, was of a rather different
character, and is discussed in a later section.
A more precise description of the criteria by which the above lists were
6of12 list word selection
- * The 6of12 list contains all non-excluded words and phrases which
- appear in 6 or more of the source dictionaries.
- * Prefixes and suffixes are excluded. Abbreviations are included;
- however, if they are entirely lower-case and alphabetic, they are
- terminated with a colon (":") so they can be easily distinguished
- from regular words.
- * Inflections of included words are not themselves included unless
- they are separately defined or irregular.
- * It sometimes occurs that a word is listed in several forms (e.g.,
- with and without hyphenation) in 6 or more dictionaries, even though
- no single form is so listed. In this case, if one spelling is
- clearly more accepted, this spelling and this spelling only is
- listed. If all spellings seem equally accepted, one spelling has
- been selected arbitrarily for inclusion.
- * The 6of12 list contains a significant number of words which do not
- meet either criterion 1 or 4 above. These words, sometimes called
- "signature words", are discussed below. All of these words are
- listed in at least one of the source dictionaries.
- * In addition to the ":" suffix discussed above, other special suffix
- characters are used to mark words with certain characteristics, as
- discussed below.
+ • The 6of12 list contains all non-excluded words and phrases which appear in
+ 6 or more of the source dictionaries.
+ • Prefixes and suffixes are excluded. Abbreviations are included; however, if
+ they are entirely lower-case and alphabetic, they are terminated with a
+ colon (":") so they can be easily distinguished from regular words.
+ • Inflections of included words are not themselves included unless they are
+ separately defined or irregular.
+ • It sometimes occurs that a word is listed in several forms (e.g., with and
+ without hyphenation) in 6 or more dictionaries, even though no single form
+ is so listed. In this case, if one spelling is clearly more accepted, this
+ spelling and this spelling only is listed. If all spellings seem equally
+ accepted, one spelling has been selected arbitrarily for inclusion.
+ • The 6of12 list contains a significant number of signature words, as
+ discussed below. All of these words are listed in at least one of the
+ source dictionaries.
+ • In addition to the ":" suffix discussed above, other annotations are used
+ to mark words with certain characteristics, as discussed below.
2of12 list word selection
- * The 2of12 list contains all non-excluded words which appear in at
- least 2 of the source dictionaries.
- * This list excludes capitalized words, multiword phrases, and
- abbreviations, as well as prefixes and suffixes. It does not exclude
- hyphenated words or contractions. If a word occurs in both a
- hyphenated and an unhyphenated form, the unhyphenated form is
- listed, even if the hyphenated form is generally preferred.
- * The list excludes spellings which are considered (by a majority of
- the dictionaries listing it) to be non-American usage. It also
- excludes secondary spellings which are mentioned by fewer than four
+ • The 2of12 list contains all non-excluded words which appear in at least 2
of the source dictionaries.
- * Inflections of included words are not themselves included unless
- they are separately defined, or irregular.
- * Several of the source dictionaries include listings for obscure
- currencies, such as ringgit, khoum and ngwee. I was unable to regard
- such words as part of the English "core vocabulary", and so I
- required citation in over a third of the dictionaries for inclusion
- of monetary units. A side-effect was the elimination of the word
- lepton, which, in addition to its use in particle physics, is also
- .01 Greek drachmas.
- * This list also includes a small number of signature words, as
- discussed below.
+ • This list excludes capitalized words, multi-word phrases, and
+ abbreviations, as well as prefixes and suffixes. It does not exclude
+ hyphenated words or contractions. If a word occurs in both a hyphenated and
+ an unhyphenated form, the unhyphenated form is listed, even if the
+ hyphenated form is generally preferred.
+ • The list excludes spellings which are considered (by a majority of the
+ dictionaries listing it) to be non-American usage. It also excludes
+ secondary spellings which are mentioned by fewer than four of the source
+ dictionaries.
+ • Inflections of included words are not themselves included unless they are
+ separately defined, or irregular.
+ • Several of the source dictionaries include listings for obscure currencies,
+ such as ringgit, khoum and ngwee. I was unable to regard such words as part
+ of the English "core vocabulary", and so I required citation in over a
+ third of the dictionaries for inclusion of such monetary units. A
+ side-effect was the elimination of the word lepton, which, in addition to
+ its use in particle physics, is also .01 Greek drachmas.
+ • This list also includes a small number of signature words, as discussed
+ below.
Signature words
-As indicated, both lists have been augmented with words (and, in the
-case of the 6of12 list, phrases) which fail to meet the formal
-requirements for inclusion. In the case of the 6of12 list, 1024 words
-were added (about 3 % of the total). These are all words which, in the
-judgment of the compiler, are as familiar as many of the words which met
-the criteria for inclusion. Examples of some of the sorts of words which
-were added are:
-
- * Words of the same category as other included words. An example is
- the astrological sign Cancer, which alone of all the astrological
- signs fails to appear in 6 or more of the dictionaries. Similarly
- added were the omitted holidays Thanksgiving and Christmas Eve.
- * Vulgarities, sexual terms and insults. Some such words were already
- included, but most of the source dictionaries were quite squeamish
- about them. These words are very widely known indeed; I hold that
- any list of "common" words which does not include the infamous
- f-word is simply discredited thereby. Some may feel that it would
- have been better to leave some or all of these terms unmentioned.
- Nevertheless, the expression of blasphemy, unwarranted contempt and
- perverse lust, whether in words or in deeds, is a very human trait.
- Suppressing the evidence of these aspects of the human condition in
- our language makes no more sense than excluding leprosy, gangrene
- and dementia, no matter how unpleasant they may be to contemplate.
- * Conventional conversational phrases so common as to be practically
- invisible to native speakers. Examples are thank you, good night,
- uh-huh, of course and gesundheit.
- * Sports terminology, especially for football and baseball. (If I, who
- am practically sports-blind, noticed this deficiency, it must be of
- major proportions indeed.)
-
-Note that the signature words in the 6of12 list can be identified via
-the suffix character "+", and eliminated if desired.
-
-A much smaller set of words (49) was added to the 2of12 list. These were
-of two sorts:
-
- * Signature words from the 6of12 list which were not already present
- in the 2of12 list, and which are not excluded due to being
- abbreviations, phrases, etc.
- * Inflections of irregular verbs not explicitly mentioned in 2 source
+As indicated, both lists have been augmented with words (and, in the case of
+the 6of12 list, phrases) which fail to meet the formal requirements for
+inclusion. In the case of the 6of12 list, 1024 words were added (about 3 % of
+the total). These are all words which, in the judgment of the compiler, are as
+familiar as many of the words which did meet the criteria for inclusion.
+Examples of some of the sorts of words which were added are:
+
+ • Words of the same category as other included words. An example is the
+ astrological sign Cancer, which alone of all the astrological signs fails
+ to appear in 6 or more of the dictionaries. Similarly added was the omitted
+ holiday Christmas Eve.
+ • Vulgarities, sexual terms and insults. Some such words were already
+ included, but most of the source dictionaries were quite squeamish about
+ them. These words are very widely known indeed; I hold that any list of
+ "common" words which does not include the infamous f-word is simply
+ discredited thereby. Some may feel that it would have been better to leave
+ some or all of these terms unmentioned. Nevertheless, the expression of
+ blasphemy, unwarranted contempt and perverse lust, whether in words or in
+ deeds, is a very human trait. Suppressing the evidence of these aspects of
+ the human condition in our language makes no more sense than excluding
+ leprosy, gangrene and dementia, no matter how unpleasant they may be to
+ contemplate.
+ • Conventional conversational phrases so common as to be practically
+ invisible to native speakers. Examples are thank you, good night, uh-huh,
+ of course and gesundheit.
+ • Sports terminology, especially for football and baseball. (If I, who am
+ practically sports-blind, noticed this deficiency, it must be of major
+ proportions indeed.)
+
+Note that the signature words in the 6of12 list can be identified via the
+annotation "+", and eliminated if desired.
+
+A much smaller set of words (49) was added to the 2of12 list. These were of two
+sorts:
+
+ • Signature words from the 6of12 list which were not already present in the
+ 2of12 list, and which are not excluded due to being abbreviations, phrases,
+ etc.
+ • Inflections of irregular verbs not explicitly mentioned in 2 source
dictionaries, such as outfought and reheard.
+These words are not marked with suffix characters.
+
Annotations
-Some of the 6of12 list entries are annotated with a suffix character,
-giving additional information about the associated word. The annotations
-can be easily removed with an editor or script if they are unwanted.
+Some of the 6of12 list entries are annotated with a suffix character, giving
+additional information about the associated word. The annotations can be easily
+removed with an editor or a script if they are unwanted.
These annotations are:
-: The word is an otherwise unmarked abbreviation. This suffix may appear
- in combination with another suffix.
-& The word is primarily a non-American usage.
-# The word is generally held to be a variant or less preferred form of
- another word.
-< This form of a word is held to be the primary form by fewer
- dictionaries than some other form of the word.
-^ This form of the word was selected arbitrarily from a set of variants,
- none of which was clearly preferred.
-= Roughly, this indicates a "second class" word, as described below.
-+ The word is a signature word.
+: The word is an otherwise unmarked abbreviation. This suffix always occurs
+ before any other suffix.
+& The word is primarily a non-American usage.
+# The word is generally held to be a variant or less preferred form of another
+ word.
+= Roughly, this indicates a "second class" word, as described below.
+< This form of a word is held to be the primary form by fewer dictionaries than
+ some other form of the word.
+^ This form of the word was selected as the most commonly listed of a set of
+ variant spellings.
+~ This form of a word is one of a set of variant spellings, none of which was
+ clearly preferred.
++ The word is a signature word.
The reasons a word might be marked with the = annotation are:
- * The word is an inflection which was defined in the same entry as the
- base word.
- * The word is a derived word (-ly, -ness or -er/or) which was not
- defined in a separate entry.
- * The word appeared in a list of undefined words with a common prefix,
- such as un- or re-.
+ • The word is an inflection which was defined in the same entry as the base
+ word.
+ • The word is a derived word (usually ending with -ly, -ness or -er/or) which
+ was not defined in a separate entry.
+ • The word appeared in a list of undefined words with a common prefix, such
+ as un- or re-.
+
+Note that, in the determination of the "<", "^", and "^" suffixes, only certain
+very close spelling variations are considered, namely single word vs.
+hyphenated word vs. multi-word, differences in capitalization, and presence or
+absence of a terminating period for abbreviations. The words tenderhearted and
+tender-hearted are close variants by this definition, but judgment and
+judgement are not.
The words in the 2of12 list are not annotated.
The 2of12inf list
-The 2of12inf list is of a rather different character from the two
-original "classic" lists. Conceptually, it is simple. It consists of all
-the words in the 2of12 list, plus their inflections, amounting to about
-81,000 words. This list may be more useful than the other lists for
-applications like word games. It was created to help Kevin Atkinson in
-his Aspell and SCOWL projects (for which, follow this link). Unlike the
-6of12 and 2of12 lists, this list is not based exclusively on the
-contents of my 12 source dictionaries, and for this reason it has, I
-feel, less authority than the other classic 12dicts lists. It also
-probably has a significantly higher error rate than the other lists, for
-reasons explained below.
+The 2of12inf list is of a rather different character from the two original
+"classic" lists. Conceptually, it is simple. It consists of all the
+unhyphenated words in the 2of12 list, plus their inflections, amounting to
+about 82,000 words. This list may be more useful than the other lists for
+applications like word games. It was created to help Kevin Atkinson in his
+Aspell and SCOWL projects (for which, follow these links). Unlike the 6of12 and
+2of12 lists, this list was not based exclusively on the contents of my 12
+source dictionaries, and for this reason it has, I feel, less authority than
+the other classic 12dicts lists. It also probably has a significantly higher
+error rate than the other lists, for reasons explained below.
The criteria defining the 2of12inf list are as follows:
- * The 2of12inf list contains all non-excluded words which appear in at
- least 2 of the source dictionaries.
- * This list excludes capitalized words, multiword phrases,
- abbreviations, contractions, hyphenated words and single-letter
- words, as well as prefixes and suffixes.
- * The list does not exclude secondary spellings, non-American usages
- or monetary units.
- * The list includes inflections of all included words. Any inflection
- mentioned or clearly implied by any of the source dictionaries is
- included (i.e., two citations are not required). Additionally, some
- inflections have been added from other sources.
- * Plurals of "uncountable" nouns were included, annotated with the "%"
- suffix character. See below for an extended discussion of the
- inclusion of these words.
- * Signature words from the other lists, plus their inflections, were
- added. No other signature words were added.
-
-Though the 2of12inf list still consists mostly of very common words,
-criteria 3 through 5 above cause the 2of12inf list to contain a greater
-proportion of unfamiliar and unusual words than the other classic
-12dicts lists.
-
-The 2of12inf list was not derived directly from the 12 source
-dictionaries. The starting point was a subset of Kevin Atkinson's AGID
-list, a list of words, parts of speech and inflections derived from
-public-domain sources, notably Moby Words and WordNet. (See the file
-agid.txt in the 12dicts archive, which is a copy of the AGID "readme",
-for more information on the antecedents of AGID.) 2of12inf was created
-by a process of editing the AGID subset to remove spurious entries and
-those which reflected a more esoteric English vocabulary than the other
-12dicts lists, and to add inflections which AGID failed to identify.
-This process required significantly less effort than would have been
-needed to derive the list directly from the source dictionaries.
-Unfortunately, a side effect of the process is that the result is likely
-to be somewhat less reliable than the other 12dicts lists. In
-particular, Moby Words is notoriously unreliable, and I find it unlikely
-that I have successfully identified all the spurious inflections its use
-has introduced. It is my hope in the future to release another edition
-of 2of12inf which is not derived from AGID, and therefore not "infected"
-by Moby Words.
-
-After the first version of the 2of12inf list was released, I replaced
-one of the source dictionaries, officially an international dictionary
-but in actuality rather British in its orientation, with a more American
-dictionary by the same publisher. It was not practical (nor necessarily
-desirable) for me to go through the list removing inflections endorsed
-only by the superseded dictionary. For this reason, the 2of12inf list
-has a slightly more international character than the other 12dicts
-lists. It is not altogether clear that this is a bad thing.
+ • The 2of12inf list contains all non-excluded words which appear in at least
+ 2 of the source dictionaries.
+ • This list excludes capitalized words, multi-word phrases, abbreviations,
+ contractions, hyphenated words and single-letter words, as well as prefixes
+ and suffixes.
+ • The list does not exclude secondary spellings, non-American usages or
+ monetary units.
+ • The list includes inflections of all included words. Any inflection
+ mentioned or clearly implied by any of the source dictionaries is included
+ (i.e., two citations are not required). Additionally, some inflections have
+ been added from other sources.
+ • Plurals of "uncountable" nouns were included, annotated with the "%" suffix
+ character. See below for an extended discussion of the inclusion of these
+ words.
+ • Qualifying signature words from the other lists, plus their inflections,
+ were added. No other signature words were added.
+ • Qualifying neologisms from the neol2016 list, including their inflections,
+ were added. The neologisms are indicated by a '!' prefix.
+
+Though the 2of12inf list still consists mostly of very common words, criteria 3
+through 5 above cause the 2of12inf list to contain a greater proportion of
+unfamiliar and unusual words than the other classic 12dicts lists.
+
+The 2of12inf list was not derived directly from the 12 source dictionaries. The
+starting point was a subset of Kevin Atkinson's AGID list, a list of words,
+parts of speech and inflections derived from public-domain sources, notably
+Moby Words and WordNet. (See the file agid.txt in the 12dicts archive, which is
+a copy of the AGID "readme", for more information on the antecedents of AGID.)
+2of12inf was created by a process of editing the AGID subset to remove spurious
+entries and those which reflected a more esoteric English vocabulary than the
+other 12dicts lists, and to add inflections which AGID failed to identify. This
+process required significantly less effort than would have been needed to
+derive the list directly from the source dictionaries. Unfortunately, a side
+effect of the process was that the result is probably somewhat less reliable
+than the other 12dicts lists. In particular, Moby Words is notoriously
+unreliable, and I find it unlikely that I have successfully identified all the
+spurious inflections its use has introduced. It would be nice to release
+another edition of 2of12inf which is not derived from AGID, and therefore not
+"infected" by Moby Words, but I haven't done so in 15 years, and so it probably
+won't happen.
+
+After the first version of the 2of12inf list was released, I replaced one of
+the source dictionaries, officially an international dictionary but in
+actuality rather British in its orientation, with a more American dictionary by
+the same publisher. It was not practical (nor necessarily desirable) for me to
+go through the list removing inflections endorsed only by the superseded
+dictionary. For this reason, the 2of12inf list has a slightly more
+international character than the other 12dicts lists. It is not altogether
+clear that this is a bad thing.
Selection of inflections
-Ideally, the 2of12inf list would contain only inflections listed in one
-of the 12dicts source dictionaries. This proved not to be practical. The
-reason for this has to do with the nature of these sources, which are
-mostly ESL dictionaries. An ESL dictionary might well list the word
-esophagus, but, because an English learner is unlikely to need to talk
-about this organ in the plural, it will probably not bother to list the
-plural form esophagi. For words of this sort, I therefore needed to
-obtain their inflections from other sources. Obviously, the decisions on
-when to include additional inflections were judgment calls, as were the
-choices of which inflections to add.
+Ideally, the 2of12inf list would contain only inflections listed in one of the
+12dicts source dictionaries. This proved not to be practical. The reason for
+this has to do with the nature of these sources, which are mostly ESL
+dictionaries. An ESL dictionary might well list the word esophagus, but,
+because an English learner is unlikely to need to talk about this organ in the
+plural, it will probably not bother to list the plural form esophagi. For words
+of this sort, I therefore needed to obtain their inflections from other
+sources. Obviously, the decisions on when to include additional inflections
+were judgment calls, as were the choices of which inflections to add.
Adjectival inflections (comparatives and superlatives) proved to be an
-especially annoying problem. Only 2 of my 12 source dictionaries
-provided remotely reliable information of this sort. In fact, such
-information is sparse and inconsistent in most dictionaries of any size.
-I relied on a small set of additional dictionaries for this information,
-which was mostly disjoint from the sources for plurals and verb forms.
-Several of these sources were Scrabble(r)-related, and therefore
-inclined to include forms of little plausibility such as iller/illest or
-fertiler/fertilest. Accordingly, I ended up rejecting some of the
-documented inflections on grounds of implausibility. I have no doubt
-that, in the process, I made a number of errors of both inclusion and
-exclusion and, in any case, many of the forms listed have no connection
+especially annoying problem. Only 2 of my 12 source dictionaries provided
+remotely reliable information of this sort. In fact, such information is sparse
+and inconsistent in most dictionaries of any size. I relied on a small set of
+additional dictionaries for this information, which was mostly disjoint from
+the sources for plurals and verb forms. Several of these sources were
+Scrabble®-related, and therefore inclined to include forms of little
+plausibility such as iller/illest or fertiler/fertilest. Accordingly, I ended
+up rejecting some of the documented inflections on grounds of implausibility. I
+have no doubt that, in the process, I made a number of errors of both inclusion
+and exclusion and, in any case, many of the forms listed have no connection
with any of the 12dicts source dictionaries.
One additional problem in the creation of the 2of12inf list was that of
-"uncountable" nouns and their plurals. Some English dictionaries,
-especially ESL dictionaries, as well as other linguistic sources attest
-to the existence of nouns which cannot be counted, or used in the
-plural. Examples of such nouns include mud, rayon, oregano, chess,
-fairness, wisdom, aluminum, training, materialism and chickenpox. This
-is an entirely commonsense notion, but a difficulty is the fact that the
-boundary between the countable and the uncountable is extremely vague
-and ill-defined. For example, the word coffee is ordinarily uncountable,
-but not when ordering in a restaurant, as is the word symmetry, except
-in physics or math. In general, it is possible to contrive a context
-where use of the plural of any noun whatsoever is reasonable.
-
-An alternate position, therefore, is that in fact no nouns are
-uncountable, and that any noun which is not already plural possesses a
-plural. This position is especially useful in the context of word games,
-where words such as zeals and anthraxes may produce large scores. For
-this reason, the official Scrabble dictionaries list words such as
-thens, onces and mankinds, which most people find rather implausible.
-The fact that the 2of12inf list might well be useful in gaming contexts,
-together with the fact that the boundary between countable and
-uncountable nouns is so ill-defined, served as a powerful argument for
+"uncountable" nouns and their plurals. Some English dictionaries, especially
+ESL dictionaries, as well as other linguistic sources attest to the existence
+of nouns which cannot be counted or used in the plural. Examples of such nouns
+include mud, rayon, oregano, chess, fairness, wisdom, aluminum, training,
+materialism and chickenpox. This is an entirely commonsense notion, but a
+difficulty is the fact that the boundary between the countable and the
+uncountable is extremely vague and ill-defined. For example, the word coffee is
+ordinarily uncountable, but not when ordering in a restaurant, as is the word
+symmetry, except in physics or math. In general, it is possible to contrive a
+context where use of the plural of any noun whatsoever is reasonable.
+
+An alternate position, therefore, is that in fact no nouns are uncountable, and
+that any noun which is not already plural possesses a plural. This position is
+especially useful in the context of word games, where words such as zeals and
+anthraxes may produce large scores. For this reason, the official Scrabble
+dictionaries list words such as thens, onces and mankinds, which most people
+find rather implausible. The fact that the 2of12inf list might well be useful
+in gaming contexts, together with the fact that the boundary between countable
+and uncountable nouns is so ill-defined, served as a powerful argument for
inclusion of all plural forms, whether commonly used or not, while its
-derivation from ESL sources argued for including only the plurals of
-countable nouns, however distinguished.
-
-In the end, I was unable to resolve this dilemma, and adopted a
-compromise. The 2of12inf list includes all plurals, but with the plurals
-of uncountable nouns marked, making it easy to remove them if they are
-not wanted. That left the issue of how to establish countability. Six of
-my source dictionaries included information on countability, which was
-adequate to decide the status of most of the included nouns. As for the
-rest, as usual, I used my best judgment. I will confess to occasionally
-overriding the source dictionaries when I believed they were clearly
-incorrect. (For instance, I chose not to mark the word hatreds as an
-uncountable plural, in defiance of the opinion of all my sources, on the
-grounds that it has been used in too many news stories from Bosnia to be
-considered unusual.) It is interesting to note that most of the plurals
-I added from auxiliary sources were of words considered uncountable.
-
-The difficulties listed above, and the fact that I was forced to
-exercise personal judgment frequently in creating it, emphasizes a
-fundamental difference between this list and the other classic 12dicts
-lists. I have tried to make the 6of12 and 2of12 lists reflect only the
-source dictionaries, and to keep my own judgments and opinions out of
-the picture (except for my addition of signature words). This has proved
-impossible to achieve for the 2of12inf list, which accordingly
-represents a less authoritative and more arbitrary collection.
-Additionally, the 2of12inf list has undergone less proofreading and
-validation than the other lists, and I suspect the error rate is
-considerably higher than the idealistic goal of 0.02 % I advocate
-elsewhere in this document. Nevertheless, I hope it may prove to be of
-some use and interest.
-
-I wish to offer my special thanks to Kevin Atkinson, for supplying me
-with the AGID list, and for encouraging me to add the inflections. Of
-course, any errors that remain in the 2of12inf list are my own
-responsibility, and should not be blamed on Kevin, AGID, or even on
-Moby.
+derivation from ESL sources argued for including only the plurals of countable
+nouns, however distinguished.
+
+As I prepared the list for release, I was unable to resolve this dilemma, and
+adopted a compromise. The 2of12inf list includes all plurals, but with the
+plurals of uncountable nouns marked, making it easy to remove them if they are
+not wanted. That left the issue of how to establish countability. Six of my
+source dictionaries included information on countability, which was adequate to
+decide the status of most of the included nouns. As for the rest, as usual, I
+used my best judgment. I will confess to occasionally overriding the source
+dictionaries when I believed they were clearly incorrect. (For instance, I
+chose not to mark the word hatreds as an uncountable plural, in defiance of the
+opinion of all my sources, on the grounds that it has been used in too many
+news stories from Bosnia to be considered unusual.) It is interesting to note
+that most of the plurals I added from auxiliary sources were of words
+considered uncountable. I also note that at some point after the release of the
+2of12inf list, I decided that it would have been better to have left the
+Scrabble plurals out, and, while I was not comfortable with removing them, no
+list I've created since then which lists inflections includes them.
+
+The difficulties listed above, and the fact that I was forced to exercise
+personal judgment frequently in creating it, emphasizes a fundamental
+difference between this list and the other classic 12dicts lists. I have tried
+to make the 6of12 and 2of12 lists reflect only the source dictionaries, and to
+keep my own judgments and opinions out of the picture (except for my addition
+of signature words). This has proved impossible to achieve for the 2of12inf
+list, which accordingly represents a less authoritative and more arbitrary
+collection. Additionally, the 2of12inf list has undergone less proofreading and
+validation than the other lists, and I suspect the error rate is somewhat
+higher than the idealistic goal of 0.02% I adopted for this project.
+Nevertheless, I hope it may prove to be of some use and interest.
+
+I wish to offer my special thanks to Kevin Atkinson, for supplying me with the
+AGID list, and for encouraging me to add the inflections. Of course, any errors
+that remain in the 2of12inf list are my own responsibility, and should not be
+blamed on Kevin, AGID, or even on Moby.
The 3esl list
The 3esl list represents another attempt to produce an English "core
-vocabulary" list. It is about 2/3 of the size of the 6of12 list, which
-it resembles in terms of the sorts of words included.
-
-The 3esl list is a far more subjective list than any of the classic
-12dicts lists. It was compiled from 3 small ESL dictionaries, using the
-same criteria for eligibility as the 6of12 list. I started with a list
-composed of all words from the smallest of the 3 sources, plus all words
-contained in both of the others. This list was then edited in the
-following ways:
-
- 1. I removed alternate spellings for included words, such as grey and
- off-stage. I also removed very similar synonyms for the same
- concept, for instance, removing cable television as a duplicate of
- cable TV.
- 2. I added one form of each word which would have been included if the
- sources had agreed on spelling, such as shortchange and back seat.
- 3. I removed some words which were present in the smallest of the
- sources but seemed too esoteric, such as the symbols of chemical
- elements. I did this only for words which were not present in the
- other sources.
+vocabulary" list. It is about 2/3 of the size of the 6of12 list, which it
+resembles in terms of the sorts of words included.
+
+The 3esl list is a far more subjective list than any of the classic 12dicts
+lists. It was compiled from 3 small ESL dictionaries, using the same criteria
+for eligibility as the 6of12 list. I started with a list composed of all words
+from the smallest of the 3 sources, plus all words contained in both of the
+others. This list was then edited in the following ways:
+
+ 1. I removed alternate spellings for included words, such as grey and
+ off-stage. I also removed very similar synonyms for the same concept, for
+ instance, removing cable television as a duplicate of cable TV.
+ 2. I added one form of each word which would have been included if the sources
+ had agreed on spelling, such as shortchange and back seat.
+ 3. I removed some words which were present in the smallest of the sources but
+ seemed too esoteric, such as the symbols of chemical elements. I did this
+ only for words which were not present in the other sources.
4. I added some words which were present in only one of the two larger
- sources, but which seemed appropriate to add. These words were
- frequently of the sort added to the 6of12 list as signature words,
- as well as some inflections that often function as words with
- meanings of their own, such as comforting and notes.
-
-All of these changes were quite subjective in nature, and quite
-numerous. Probably more than 10 % of the candidate words were added or
-removed in this way. For this reason, it is pointless to speak of
-signature words for this list; the composition of the list is too
-arbitrary for the term to make any sense. (I will note that the list is
-still not entirely arbitrary, as I added only words found in some form
-in one of the sources, and removed no words present in two of the
-sources other than duplicates. Thus, words like front page were not
-added, no matter how familiar, and words such as lugubrious were not
-removed, despite clearly not being part of any "core vocabulary".)
-
-Like the 6of12 list, the 3esl list marks lower-case abbreviations with a
-":" suffix, to prevent them from being mistaken for regular English
-words.
+ sources, but which seemed appropriate to add. These words were frequently
+ of the sort added to the 6of12 list as signature words, as well as some
+ inflections that often function as words with meanings of their own, such
+ as comforting and notes.
+
+All of these changes were quite subjective in nature, and quite numerous.
+Probably more than 10 % of the candidate words were added or removed in this
+way. For this reason, it is pointless to speak of signature words for this
+list; the composition of the list is too arbitrary for the term to make any
+sense. (I will note that the list is still not entirely arbitrary, as I added
+only words found in some form in one of the sources, and removed no words
+present in two of the sources other than duplicates. Thus, words like front
+page were not added, no matter how familiar, and words such as lugubrious were
+not removed, despite clearly not being part of anyone's "core vocabulary".)
+
+Like the 6of12 list, the 3esl list marks lower-case abbreviations with a ":"
+suffix, to prevent them from being mistaken for regular English words.
One final note on this list. The 3esl list contains about 1500 words not
-present in the 6of12 list. Because these two lists have the same rules
-for the kinds of words included, one could easily combine the two to
-produce a slightly larger list including a number of words whose
-omission from 6of12 is rather surprising. Be warned that in a few cases,
-the spelling chosen for words with multiple spellings is different in
-the two lists, and I would recommend that the duplicates be removed.
-(I'll be happy to provide a list of the duplicates if anyone wants one.)
+present in the 6of12 list. Because these two lists have the same rules for the
+kinds of words included, one could easily combine the two to produce a slightly
+larger list including a number of words whose omission from 6of12 is rather
+surprising. Be warned that in a few cases, the spelling chosen for words with
+multiple spellings is different in the two lists, and I would recommend that
+the duplicates be removed. (I'll be happy to provide a list of the duplicates
+if anyone wants one.)
+
+The international 12dicts lists
+
+Four 12dicts lists contain a more cosmopolitan vocabulary than the classic
+lists. Two of these lists, 2of4brif and 5d+2a (previously called 5desk), were
+released over ten years ago. The 2of4brif list was derived from four British
+dictionaries, and has now been deprecated, as I believe the 3of6game list to be
+a superior implementation of the same concept, compiled from more recent
+sources. The 5d+2a list was originally compiled from a variety of sources, but
+was extensively revised for this release by addition of several fairly recently
+published sources.
+
+For release 6, two new international lists were added to 12dicts: 3of6game and
+3of6all. These were based on 6 "advanced learner's" ESL dictionaries, released
+by both American and British publishers, most of which covered both strains of
+English. The 3of6game list is intended primarily for use in word games, and can
+be compared to 2of12inf in its general approach. The 3of6all list includes more
+forms of words (hyphenated, capitalized, multi-word phrases, etc.), and can be
+compared to 6of12 in its general approach.
+
+Two other more unusual lists were derived from these sources: 6phrase and
+2of5core. 6phrase is a collection of all the multi-word phrases from any of the
+six dictionaries. Five of the six international sources flag some words as
+being the most important words for an English beginner to master. The 2of5core
+list collects those words that are flagged in at least two of these
+dictionaries. Both of these lists are discussed in a little more detail in the
+"Specialized Lists" section of this document.
+
+The following table summarizes the contents of each of the lists in the
+International directory, ordered by size in words:
+┌─────────────────┬────────┬────────┬────────────┬───────┐
+│ │2of4brif│3of6game│ 5d+2a │3of6all│
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Size (Words) │60,000 │65,000 │68,000 │83,000 │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Number of Sources│4 │6 │7 (+5 minor)│6 │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│American English │Some │Y │Y │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│British English │Y │Y │Y │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Ordinary words │Y │Y │Y │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Inflections │Y │Y │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Hyphenations │– │– │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Phrases │– │– │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Names │– │– │Y │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Abbreviations │– │– │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Acronyms │– │– │Y │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Prefixes/Suffixes│– │– │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Signature words │– │Y │– │Y │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Neologisms │– │Y │– │– │
+├─────────────────┼────────┼────────┼────────────┼───────┤
+│Annotations │– │Y │– │Y │
+└─────────────────┴────────┴────────┴────────────┴───────┘
The 2of4brif list
-All of the classic 12dicts lists are unabashedly oriented towards
-American English. I've received a few expressions of interest in a
-British English list. The result is the 2of4brif list. This list was
-compiled from 4 large "international" ESL dictionaries, published by
-British publishers. To this American, they are more British than they
-are international; quite possibly, they seem more American than
-international to British readers. It is interesting to note that,
-although there were only a third as many sources for this list as for
-the 12dicts lists, these dictionaries resembled each other far more
-closely than their American counterparts, which could mean that the
+All of the classic 12dicts lists are unabashedly oriented towards American
+English. After receiving a few expressions of interest in a British English
+list, I put together the 2of4brif list. This list was compiled from 4 large
+"international" ESL dictionaries, published by British publishers. To this
+American, they are more British than they are international; quite possibly,
+they seem more American than international to British readers. It is
+interesting to note that, although there were only a third as many sources for
+this list as for the 12dicts lists, these dictionaries resembled each other far
+more closely than their American counterparts, which could mean that the
2of4brif list is as good an approximation of a "core" British English
-vocabulary as the 6of12 list is for American English. (Or, alternately,
-it may simply mean that my choice of sources was too narrow.)
-
-This criteria for inclusion in this list were basically those of the
-2of12inf list. In particular, inflections are included for all words,
-but hyphenated words, contractions, phrases, proper names and
-abbreviations are all excluded. One important difference between the two
-is the way in which inflections were determined for inclusion. The
-2of12inf list includes some inflections found in one (or even none) of
-its sources. Further, as discussed in detail above, it includes plurals
-for words which are not normally considered to have plurals. The
-2of4brif list differs in both of these regards. It includes only
-inflections endorsed by two or more of the sources, specifically
-excluding any plural forms for nouns listed as uncountable.
-
-The 2of4brif list includes no signature words as such. I made a small
-number of adjustments for consistency, such as making sure that -ise and
--ize spellings were equally represented, and adding plurals for ordinal
-numbers. (Why fourteenth would be defined as a fraction, but not
-seventeenth, I must simply regard as a mystery.) These edits were so
-few, and so clearly harmless, that I have not marked them.
-
-Prospective users of the 2of4brif list should realize that it was
-compiled by an American. If my sources contained any glaring errors (and
-most dictionaries have a few), I might well not have noticed, and
-perpetuated them in the list. The fact that two citations were required
-is some protection against such an event, but no guarantee.
-
-As the 2of4brif list is very similar in makeup to the 2of12inf list, a
-user who wants a larger, more international list than either could
-reasonably merge the two. If you do this, you should remove the unusual
-plurals (marked with a "%") from the 2of12inf list in the process, for
-consistency.
-
-The 5desk list
-
-I created the 5desk list in an attempt to do a better /usr/dict/words
-(about which I offer many harsh criticisms elsewhere in this document).
-The sorts of words admitted are the same sorts that /usr/dict/words
-contains. Though somewhat larger in size than most versions of /usr/dict
-/words, this is still a short word list, striving for inclusion of words
-one is likely to encounter rather than the complete jargon of every
-possible scientific, artistic or occult endeavor.
-
-5desk was assembled primarily from five "desk dictionaries". It was
-augmented by words from five minor sources, including a "vocabulary
-builder" and a collection of proper names. The list excludes prefixes,
-suffixes, phrases, hyphenated words, contractions and most abbreviations
-and acronyms. There was no requirement for multiple listings; all
-qualifying words from each of the sources were included. Inflections of
-included words were not included themselves except when irregular, or
-separately defined. Variant and non-American spellings were not
-excluded, and no signature words were added.
-
-Words commonly considered to be abbreviations/acronyms were included if
-they contained at least one upper case character, and were defined with
-an explicit part of speech. This excluded items like Mr and Feb, which
-are abbreviations in the classic sense, but allowed words like DNA and
-ATM, which are used far more frequently than that which they abbreviate.
-While there is a trend in modern dictionaries to list such words as
-nouns (or occasionally verbs, adverbs, etc.), it is a trend in progress,
-and rather inconsistently applied. For this reason, the set of such
-words in the 5desk list is somewhat incoherent, including SPCA but not
-PETA, AIDS but not SIDS, KGB but not CIA, and PDQ but not ASAP.
-
-One class of commonly-used words is regrettably absent from the 5desk
-list, because I was unable to find a satisfactory source for them. This
-is the class of commercial names such as Exxon, Tylenol, Pepsi and Chevy
-. This is probably forgivable, as this class of names is as ephemeral
-and transitory as teenage slang. The one-time household words Kool,
-Ovaltine, Philco and Ipana serve now only as answers to trivia
-questions, with modern wonders like Starbucks, Google, Ritalin and TiVo
-taking their place on the tongues of the trendy.
-
-The 5desk list has clearly moved beyond any "core vocabulary" concept.
-It includes quite esoteric words (ogee, pleonastic), very uncommon
-spellings (thiamine, yuppy), and obscure geographical and historical
-names (Paricutin, Nevelson). Like /usr/dict/words, it is frequently
-inconsistent and arbitrary, but I hope at the least I have avoided
-including spelling errors, and overlooking the stuff of everyday
-conversation. Perhaps it will be useful as a compromise between basic
-lists such as 3esl, and truly massive lists like Mendel Cooper's ENABLE.
+vocabulary as the 6of12 list is for American English. (Or, alternately, it may
+simply mean that my choice of sources was too narrow.)
+
+This criteria for inclusion in this list were basically those of the 2of12inf
+list. In particular, inflections are included for all words, but hyphenated
+words, contractions, phrases, proper names and abbreviations are all excluded.
+One important difference between the two is the way in which inflections were
+determined for inclusion. The 2of12inf list includes some inflections found in
+one (or even none) of its sources. Further, as discussed in detail above, it
+includes plurals for words which are not normally considered to have plurals.
+The 2of4brif list differs in both of these regards. It includes only
+inflections endorsed by two or more of the sources, specifically excluding any
+plural forms for nouns listed as uncountable.
+
+The 2of4brif list includes no signature words as such. I made a small number of
+adjustments for consistency, such as making sure that -ise and -ize spellings
+were equally represented, and adding plurals for ordinal numbers. (Why
+fourteenth would be defined as a fraction, but not seventeenth, I must simply
+regard as a mystery.) These edits were so few, and so clearly harmless, that I
+have not marked them.
+
+Prospective users of the 2of4brif list should realize that it was compiled by
+an American. If my sources contained any glaring errors (and most dictionaries
+have a few), I might well not have noticed, and perpetuated them in the list.
+The fact that two citations were required is some protection against such an
+event, but no guarantee.
+
+As the 2of4brif list is very similar in makeup to the 2of12inf list, a user who
+wants a larger, more international list than either could reasonably merge the
+two. If you do this, you should remove the unusual plurals (marked with a "%")
+from the 2of12inf list in the process, for consistency.
+
+Note that I have deprecated the 2of4brif list. I believe that any applications
+of this list would be better off using the 3of6game list in its place.
+
+The 3of6 lists
+
+The lists 3of6game and 3of6all are new with version 6 of 12dicts. Both were
+derived from a set of six advanced learner's ESL dictionaries. The dictionaries
+can be broken down as follows:
+
+ • One strongly American-oriented dictionary.
+ • Two somewhat British-oriented dictionaries.
+ • Three international dictionaries, one from an American publisher, two from
+ a British publisher.
+
+This provided a good balance between British and American usage. My goal was to
+produce lists that contained blancmange and swede as well as applesauce and
+boysenberry. Note that some of the British dictionaries include words from
+Australian, Indian, African and Caribbean English, and a fraction of this
+vocabulary made it into the 3of6 lists.
+
+In previous versions of 12dicts, I asked users to tell me what they were doing
+with the lists. The most common answer was that they were used to supply the
+vocabulary for a word game. The 3of6game list was designed to fulfill this
+purpose. It contains only the sort of words likely to be used in a word game
+(no hyphenated words, proper names, abbreviations, contractions or phrases),
+but does contain inflections. In general, words must appear in three of the
+sources to be included. The rules, however, do provide for a number of
+(annotated) exceptions, including uncommon inflections and words whose most
+common form is either hyphenated or phrasal. Details are below.
+
+The 3of6all list is a larger list, basically containing any kind of word you
+can imagine, if found in three of the sources. As with 3+3game, some additional
+words were added as exceptions, but there are not as many of them, as the goal
+of this list is to be as faithful as reasonable to the sources.
+
+Both the 3of6game and 3of6all lists contain signature words/phrases. The
+3of6game list also contains neologisms, as game players are likely to want to
+play recently coined or popularized words.
+
+The 3of6game list
+
+The 3of6game list contains words which are listed in 3 of the 6 advanced
+learners dictionaries described above. Only words suitable for play in most
+word games are included, excluding hyphenated words, multi-word phrases,
+capitalized words, abbreviations and contractions. There are no restrictions on
+length - in particular, it contains four one-letter words: a, x (a verb meaning
+to cross out), I and O, the last two of which are included despite their
+capitalization (which is an English spelling phenomenon entirely disconnected
+from logic). In certain cases, words are present in this list despite being
+listed in fewer than three sources. This serves the purpose of offering game
+players more words in situations where lexicographers differ about what word
+forms are correct. Some exceptional situations are:
+
+ • A word is one of a set of close variants, none of which is present in three
+ of the sources. These words are marked with a "^" suffix. An example is the
+ word aqualung, which is sometimes capitalized or hyphenated.
+ • The word is a British spelling of an American word listed in three sources,
+ or an American spelling of a British word from three sources. These words
+ are marked with a "&" suffix. Examples include prolog, an American form of
+ the British prologue, and hyaena, a British spelling of the American hyena.
+ • A word is a plural of a word which only two of the sources describe as
+ countable, such as boyhoods. Similarly, adjectival inflections are added if
+ as few as two of the sources attest to it, as with frillier and frilliest.
+ • A word is an unusual inflection of a word where at least three sources
+ agree that some inflection is called for, such as the less common plural
+ planetaria of planetarium.
+ • A word is an inflection for a word used as an unusual part of speech, whose
+ meaning is closely related to a more common meaning. Examples are the verb
+ forms autopsied and autopsying, whose meanings are closely related to the
+ common meaning of the noun autopsy.
+ • A word is a unhyphenated form of a word normally hyphenated or written
+ phrasally such as ballgame, which is more commonly written ball game.
+
+Words not present in three of the source dictionaries are marked with the "$"
+suffix character if the "^" and "&" annotations do not apply.
+
+The 3of6game list includes both signature words and neologisms, marked with a
+"+" or "!" respectively. There are 520 signature words for this list,
+representing words that I feel "ought to be" included. Each signature word is
+present in at least one of the source dictionaries. Virtually all of these
+words are American English, as I am not qualified to tell whether a interesting
+Britishism like tosspot is used often enough to justify its addition as a
+signature word. Note that the presence of annotations allows a user to remove
+these extra words if she finds their addition unjustified.
+
+The 3of6game list could be combined with the 2of12inf list (minus the
+uncountable plurals) and/or 2of4brif if a larger list is required. Note that
+because 2of2inf is very strongly American, such a combination will be less
+balanced between American and British English than 3of6game itself.
+
+The 3of6all list
+
+The 3of6all list contains words which are listed in three of the six advanced
+learner's dictionaries. In contrast to the 3of6game list, no words are
+excluded, not even abbreviations, prefixes or suffixes. Most words have their
+inflections included. An exception is made for phrasal verbs and other verb
+phrases, whose inflections are completely predictable from the initial word of
+the phrase.
+
+The 3of6all list contains many phrasal verbs, such as let down, take after,
+sound off and make out, whose meanings are often quite hard for inexperienced
+students of English to guess. Phrasal verbs are marked by the ";" suffix
+character. Only four of the six source dictionaries provide phrasal verb
+information in an easy-to-collect way. For this reason, I put a phrasal verb
+into the 3of6all list even if I found it in only two of the sources.
+
+The 3of6all list contains some other words present in fewer than three of the
+dictionaries, though not as many as 3of6game. All such words are marked. The
+cases where this occurs are as follows:
+
+ • As described for the 3of6game list, a word is one of a set of close
+ variants, none of which is present in three of the sources. These words are
+ marked with a "^" suffix. For this list, in addition to differences in
+ hyphenation or single/multi-word format, variants only in capitalization or
+ (for abbreviations) the presence or absence of a period are considered
+ close.
+ • As described for the 3of6game list, a word is a British spelling of an
+ American word listed in three sources, or an American spelling of a British
+ word from three sources. These words are marked with a "&" suffix.
+ • A few other words present in fewer than three of the dictionaries are
+ added. Usually, this occurs when a word is found by three sources to have
+ the same part of speech, but the sources fail to agree on the spelling of
+ the inflection(s). An example is the word Grammy, whose plural is claimed
+ by two sources to be Grammies, and by two others to be Grammys. These words
+ are annotated with the "$" suffix.
+
+There is one other situation where an annotation suffix is used. This occurs
+when a word is shown by a majority of the sources as being used only in a few
+specific phrases, even though other dictionaries may give it a regular
+definition. An example is the word bated, which is shown by most of the sources
+as used only in the phrase with bated breath. In this case, the word is flagged
+with a ">" suffix. A search on a word so flagged will reveal the key phrase(s)
+elsewhere in the list.
+
+Recall that, sometimes, a word may have more than one suffix. An abbreviation
+shown with the ":" suffix (indicating the absence of a final period) may be
+followed by another suffix, and the combination ">^" appears upon occasion.
+
+The 3of6all list contains signature phrases, but no neologisms. The signature
+phrases are marked with the "+" suffix. The 629 3of6all signatures are all
+basic conversational idioms and common connective phrases, like I told you so,
+in front of and on the other hand. Though these phrases often show up in the
+sources in lists of idioms, they generally do not appear as separate headwords,
+which kept me from easily recording their presence. I believe, however, that
+all of these phrases are extremely common, and deserve to be included in this
+list. The signature phrases are all marked with the "+" suffix.
+
+The 5d+2a list
+
+I created the 5d+2a list (originally called 5desk) in an attempt to do a better
+/usr/dict/words (the failings of which were a large part of my motivation for
+doing 12dicts in the first place). The sorts of words admitted are the same
+sorts that /usr/dict/words traditionally contains. Though somewhat larger in
+size than many versions of /usr/dict/words, this is still a short word list,
+striving for inclusion of words one is likely to encounter rather than the
+complete jargon of every possible scientific, artistic or occult endeavor.
+
+The original 5desk list was assembled primarily from five "desk dictionaries".
+It was augmented by words from five minor sources, including a "vocabulary
+builder" and a collection of proper names. It excluded prefixes, suffixes,
+phrases, hyphenated words, contractions and most abbreviations and acronyms.
+There was no requirement for multiple listings; all qualifying words from each
+of the sources were included. Inflections of included words were not included
+themselves except when irregular, or separately defined. Variant and
+non-American spellings were not excluded, and no signature words were added.
+
+Words commonly considered to be abbreviations/acronyms were included if they
+contained at least one upper case character, and were defined with an explicit
+part of speech. This excluded items like Mr and Feb, which are abbreviations in
+the classic sense, but allowed words like DNA and ATM, which are used far more
+frequently than that which they abbreviate. While there is a trend in modern
+dictionaries to list such words as nouns (or occasionally verbs, adverbs,
+etc.), it is a trend in progress, and rather inconsistently applied. For this
+reason, the set of such words in the 5desk list is somewhat incoherent,
+including SPCA but not PETA, AIDS but not SAD, KGB but not CIA, and PDQ but not
+ASAP.
+
+When version 6 of 12dicts was released, the 5desk list was augmented by adding
+qualifying words from two advanced learner's ESL dictionaries, and as a result
+renamed to 5d+2a.txt. Both of the additional dictionaries had a strongly
+international vocabulary, causing the new list to have a less American and more
+cosmopolitan character. This increased the size of the list by about 20% to
+about 68,000 words.
+
+One class of commonly-used words is regrettably absent from the 5desk list,
+because I was unable to find a satisfactory source for them. This is the class
+of commercial names such as Exxon, Tylenol, Pepsi and Chevy. This is probably
+forgivable, as this class of names is as ephemeral and transitory as teenage
+slang. The one-time household words Kool, Ovaltine, Philco and Ipana serve now
+only as answers to trivia questions, with modern wonders like Starbucks,
+Google, Ritalin and TiVo taking their place on the tongues of the trendy.
+
+The 5d+2a list contains no signature words. I did take the liberty of adding
+the personal names of around thirty well-known individuals, mostly statesmen
+and politicians. Though the original 5desk list contained many such names from
+all periods of human history, I have not found a useful source to bring the
+list into the twenty-first century. At the same time, I felt that distributing
+a list full of names that did not include Cheney and Obama was not reasonable.
+So I compromised by adding a few names whose historical significance was clear
+to me, until such time as a better source than my own memories of the last 15
+years can be found.
+
+The 5d+2a list has clearly moved beyond any "core vocabulary" concept. It
+includes quite esoteric words (ogee, pleonastic), very uncommon spellings (
+thiamine, yuppy), and obscure geographical and historical names (Paricutin,
+Nevelson). Like the traditional /usr/dict/words, it is frequently inconsistent
+and arbitrary, but I hope at the least I have avoided including spelling
+errors, and overlooking the stuff of everyday conversation. Perhaps it will be
+useful as a compromise between basic lists such as 3esl, and truly massive
+lists like Mendel Cooper's ENABLE.
+
+The lemmatized 12dicts lists
+
+Version 6 of 12dicts provides three lemmatized lists combining words from the
+2of12inf, 3of6game and 2of4brif lists. The word "lemmatized" is a rare word,
+which you will find in none of these lists, but what it means is that these
+lists are formatted as a collection of word sets, called lemmas (or lemmata, if
+you're into irregular plurals), each set composed of a headword and some number
+(possibly zero) of closely related words. Two of these lists were introduced in
+version 5 of 12dicts, but they have undergone major revisions since then.
+
+The three lists are 2+2+3lem (originally 2+2lemma), 2+2+3frq (originally
+2+2gfreq) and 2+2+3cmn. 2+2+3lem simply arranges the words of the three source
+lists into lemmas and lists them alphabetically by headword. 2+2+3frq arranges
+the same lemmas by approximate order of their frequency of usage, computed with
+the help of a frequency list obtained from Brigham Young University (BYU),
+omitting those words and lemmas whose usage is so small that they fail to show
+up in the BYU material. 2+2+3cmn extracts a subset of the lemmas of 2+2+3lem,
+namely those lemmas with a certain minimum level of usage (approximately the
+level of the word butterscotch), and lists them alphabetically by headword.
+This is yet another attempt in 12dicts to generate a core English vocabulary.
+
+The advantage of a lemmatized presentation of words is that it puts related
+words together, even when spellings differ greatly, as for be, are, is and were
+. A moderate disadvantage is that the same word can appear in more than one
+lemma, such as putting, which is present in the lemmas headed by both put and
+putt. Overall, I find the lemmatized format to be clearer and more useful than
+a simple alphabetized list, and I rather wish I had released the other lists
+which include inflections in that format.
+
+The following table summarizes the contents of each of the lists in the
+Lemmatized directory, ordered by size in words:
+┌─────────────────┬────────┬────────┬────────┐
+│ │2+2+3cmn│2+2+3frq│2+2+3lem│
+├─────────────────┼────────┼────────┼────────┤
+│Size (Words) │25,000 │34,000 │84,000 │
+├─────────────────┼────────┼────────┼────────┤
+│Number of Sources│21 │21 │21 │
+├─────────────────┼────────┼────────┼────────┤
+│American English │Y │Y │Y │
+├─────────────────┼────────┼────────┼────────┤
+│British English │Some │Some │Y │
+├─────────────────┼────────┼────────┼────────┤
+│Ordinary words │Y │Y │Y │
+├─────────────────┼────────┼────────┼────────┤
+│Inflections │Some │Some │Y │
+├─────────────────┼────────┼────────┼────────┤
+│Hyphenations │Some │Some │Y │
+├─────────────────┼────────┼────────┼────────┤
+│Phrases │– │– │– │
+├─────────────────┼────────┼────────┼────────┤
+│Names │Some │Some │– │
+├─────────────────┼────────┼────────┼────────┤
+│Abbreviations │Some │Some │– │
+├─────────────────┼────────┼────────┼────────┤
+│Acronyms │Some │Some │– │
+├─────────────────┼────────┼────────┼────────┤
+│Prefixes/Suffixes│– │– │– │
+├─────────────────┼────────┼────────┼────────┤
+│Signature words │Y │* │* │
+├─────────────────┼────────┼────────┼────────┤
+│Neologisms │A few │A few │Y │
+├─────────────────┼────────┼────────┼────────┤
+│Annotations │Y │Y │Y │
+└─────────────────┴────────┴────────┴────────┘
+
+A * in the "Signature Words" row means that signature words associated with
+some other list may be present, but there are no signature words associated
+specifically with that list.
+
+The 2+2+3lem list
+
+The list 2+2+3lem.txt contains the words in the 2of12inf, 2of4brif and 3of3game
+lists. Also, the new words from the neol2016.txt list have been added, marked
+with a "!" if they would not have otherwise been included. (Marking the new
+words permits them to be removed if it is preferred for this list to be in
+synch with the other 12dicts lists.) Furthermore, some high-frequency
+hyphenated words from 2of12.txt and 3of6all have been added. These words were
+originally added to the lemmatized frequency list (see below), and I liked the
+results so much that I added them to this list as well. Finally, British forms
+of words in the 2of12inf list not already in the other lists have been added.
+Words marked with a % in the 2of12inf list ("Scrabble plurals") have however
+been omitted.
+
+In the previous version of 12dicts, the 2+2+3lem list was called 2+2lemma. The
+only significant changes were the addition of new words, and switching from "+"
+to "!" to mark neologisms in the list.
+
+The 2+2+3lem list is not formatted as a simple list of words. It is composed of
+entries of 1 or 2 lines each. The first line contains a headword, and the
+second line, which is indented if present, contains an alphabetized list of
+related words. A simple example:
+
+funny
+ funnier, funnies, funniest, funnily, funniness
+
+The list of related words contains three sorts of entries.
+
+ 1. Inflections.
+
+ 2. Variant spellings.
+
+ 3. Words formed with certain suffixes.
+
+In addition to true variant spellings such as grey for gray and thru for
+through, item 2 also includes words which, though pronounced differently, are
+clearly variants of the headword. Thus, hooray is considered a variant of
+hurrah (but mere synonyms like furze and gorse remain independent).
+
+Item 3 is based on a small list of suffixes, producing closely and consistently
+related words. These suffixes are -ful, -ish, -less, -like, -ly, -most and
+-ness. -ally is also allowed, if there is no -al word to apply the -ly suffix
+to. (For instance, basically is considered to be derived from basic, because
+there is no word basical.) When one of these suffixes is used in an unusual
+way, the resulting word is considered independent. For instance, likely is not
+considered to be derived from like, nor bashful from bash. There are some
+rather difficult questions here, such as how closely slavish is related to
+slave, or sluggish to slug. In general, I have chosen the course of least
+surprise by treating such pairs as independent.
+
+Here are some other notes on the determination of what words are related.
+
+Certain uses of the suffixes -ed and -s are treated as inflections, even though
+technically they are not. Thus, talented is treated as derived from talent, and
+optics from optic.
+
+Words ending with the suffix -ability/ibility are treated as relatives of the
+corresponding -able/ible word.
+
+Sometimes, the choice of which variant to treat as the headword is somewhat
+arbitrary. I have consistently chosen an American spelling over a British
+spelling here. This has some effect on the number of headwords. I treat cheque
+as a variant of check, whereas, to an observer with a British bias, they would
+no doubt be separate headwords.
+
+No distinction is made of different meanings of the same word, even when they
+are so different that dictionaries list them separately. wind the noun and wind
+the verb are considered as a single word, as are second the adjective, second
+the noun and second the verb.
+
+It may sometimes happen that two different words have the same inflection (
+putting derives both from putt and put; holier relates to holey as well as holy
+), or that an inflection is a headword in its own right (as with wound, the
+past tense of wind, or crooked, the past tense of crook). These situations are
+noted in the 2+2+3lem list as cross-references to the alternate headword. There
+are two specific situations which might not be obvious where inflections are
+treated as different words. These occur when a present tense form or a -ness
+word has a plural inflection, as with meaning and weakness. Such words are
+always made headwords, even when the relationship to the original root is very
+close. Here is an example showing how cross-references are indicated:
+
+base
+ based, baseless, basely, baseness, baser, bases -> [basis], basest, basing
+
+Almost always, a given word has only one cross-reference - the biggest
+exception is the incredible tangle shown in the example below:
+
+slue -> [slough]
+ slew -> [slay, slew, slough], slewed, slewing, slews -> [slew, slough],
+slued, slues -> [slough], sluing
+
+where 4 uncommon words mostly pronounced sloo have become thoroughly confused.
+
+The 2+2+3frq list
+
+In the previous version of 12dicts, there was a file called 2+2gfreq.txt. This
+file has been completely replaced by a new implementation of the same idea.
+Like the older list, the 2+2+3frq list presents the lemmas of 2+2+3lem in bands
+of lemmas with about the same frequency of use. However, there are the
+following major differences from what was done before:
+
+ • In the previous version, word frequency information was obtained from data
+ collected from the World Wide Web supplied by Google. This data was very
+ voluminous, but was quite distorted by the Web's emphasis on computerese,
+ pornography and marketing. I am now using a commercial word frequency
+ database, supplied by Brigham Young University, based on its Corpus of
+ Contemporary American English (COCA). This data is less voluminous than the
+ Google data, but is far more balanced and seemingly trustworthy. It has
+ some other advantages, discussed below.
+ • High-frequency hyphenated words from 2of12inf and 3of6all have been added.
+ I liked the effect of this so much that I added the same words to the
+ 2+2+3lem list.
+ • A certain number of high frequency abbreviations, contractions and
+ capitalized words were added. Some of these words were not to be found in
+ any other 12dicts list, for which reason I did not also add them to
+ 2+2+3lem.
+ • The list was shortened by omitting all lemmas which did not appear at all
+ in the BYU data.
+ • Individual lemmas were shortened by omitting very infrequent words and all
+ regular inflections, except when they were used frequently as a part of
+ speech different from the headword, such as disappointed as an adjective
+ rather than a verb form.
+
+The lemmas of 2+2+3frq are grouped into bands by the combined number of
+occurrences in the BYU data of the words in the lemmas. Band 21 contains lemmas
+whose words together appear between 16 and 31 times in the BYU data. Each other
+band contains lemmas of twice the frequency of the following band, that is,
+each lemma in band 20 appears in the BYU data between 32 and 63 times, and so
+on. The first band contains the three lemmas most frequently used in the
+English language (according to BYU), namely the, be (plus its inflections) and
+to. As already noted, some words are found in multiple lemmas. One helpful
+aspect of the BYU data is that it separates frequency data for a word by parts
+of speech, and notes the base word for inflected words. This often allows the
+frequency counts for a word like building to be accumulated under the correct
+lemma (either build or building). In the event that the BYU data is unable to
+completely resolve the appropriate lemma for a word, its frequency count is
+divided equally among the various candidates.
+
+2+2+3frq is divided into bands by lines like this:
+
+----- 5 -----
+
+The lemmas in each band are presented in alphabetical order, not by the
+frequency of the individual lemma.
+
+Note that because the BYU data was extracted from a corpus of American English,
+the 2+2+3frq file tilts in an American direction, though some British words
+like bloke, colour and lorry have made it through.
+
+A useful attribute of the BYU data is that it, unlike the Google data, includes
+hyphenated words, as well as some abbreviations, contractions and capitalized
+words. The two cases are rather different. The inclusion of hyphenated words is
+explicitly intended. However, the BYU documentation states that proper names
+have been excluded where possible, while admitting that, in many cases, the
+software processing the data was unable to be sure whether a word was a proper
+name or not, in which case the word was included. The effect is that there are
+many words generally considered to be proper names present, notably the names
+of months of the year and days of the week, plus those of religions,
+nationalities and ideologies. You will not find names like linda, picasso,
+vladivostok, microsoft or rumpelstiltskin in the data, but you will find
+november, buddhist, peruvian and marxist, to the extent that I wonder if BYU
+might have used a different definition of "proper name" than the one I was
+taught in school. As for abbreviations, the BYU documentation makes no mention
+of them, but there are some very familiar abbreviations in the data. There are
+not a lot of them, which makes me wonder whether their presence was intentional
+or a processing error. Either way, I have no reason to doubt their frequency
+counts.
+
+I decided that I wanted to add high-frequency hyphenated words, proper names
+and abbreviations to the frequency list, as I consider this data to be very
+interesting. When I did so, I discovered in band 17 the words atlantean and
+klingon. I really don't think that these words have anywhere close to the same
+frequency as armband and carpool, which are also present in band 17. This makes
+me suspect that, for words of this frequency or less, the BYU data is starting
+to become less reliable. For this reason, I decided to stop adding hyphenated
+words, capitalized words, contractions and abbreviations after band 17.
+
+In the case of hyphenated words, I added them to the 2+2+3frq list only if they
+were present in either 2of12.txt or 3of6all.txt. I also added these words to
+the 2+2+3lem list. In the case of abbreviations and capitalized words, there
+were not all that many of them, and some of them were not present in any other
+12dicts list, such as Americanist, Thatcherism and, of course, Klingon. For
+this reason, when I added capitalized words, contractions and abbreviations to
+2+2+3frq, I parenthesized them to indicate that their presence had nothing to
+do with any source but the BYU data. The same consideration led me to omit
+these words from the 2+2+3lem list.
+
+The 2+2+3frq list is considerably smaller than the previous 2+2gfreq list due
+to my decision to drop lemmas which were absent from the BYU data, especially
+since the BYU data was considerably less voluminous and so left out many more
+words than the Google data. In addition, I observed that many high-frequency
+lemmas contained unusual spellings and archaic forms that were not present in
+the BYU data, such as cocoanut, iodin and didst, and decided to drop
+non-headwords from the lemmas unless their frequency was at or above the level
+of band 17. A similar decision was made to drop regular inflections from the
+lemmas in the 2+2+3frq list unless they had high frequency with a different
+part of speech, for example, loving as an adjective or fighting as a noun.
+Finally, I chose to drop the word/lemma cross-references from the 2+2+3frq
+list, replacing them with a * indicating that a word was to be found under
+another headword (though it might have been suppressed if it was a regular
+inflection).
+
+As an example of how this works out in practice, here is the lemma for time
+from 2+2+3lem:
+
+time
+ timed, timeless, timelessly, timelessness, times, timing -> [timing]
+
+and here is the condensed version from 2+2+3frq.
+
+time
+ timed, timeless
+
+The words timelessly and timelessness are not used often enough (according to
+BYU) to mention in the frequency list, while the word times was not frequently
+used except as a form of time, and, while the word timing was frequently used
+as a noun, its counts were collected under the lemma timing rather than time.
+
+The 2+2+3cmn list
+
+The 2+2+3cmn list is a relatively simple transformation of the 2+2+3frq list,
+in yet another attempt to produce a "core English" word list. It is composed of
+the lemmas of the 2+2+3frq list from bands 1 through 17, sorted in alphabetical
+order by headword. Minor formatting differences are that the "!" is removed
+from neologisms, and the parentheses are removed from capitalized words,
+abbreviations and contractions.
+
+I have added 67 signature words to 2+2+3cmn, which are abbreviations,
+contractions and capitalized words (mostly contractions) which I know to be
+extremely high frequency, but which were not present in the BYU data, words
+such as can't, Mr. and DVD. These words are marked with a + to indicate their
+absence from the 2+2+3frq source data.
+
+Like 2+2+3frq, 2+2+3cmn tilts strongly in the direction of American English.
+
+Because all the words of 2+2+3cmn are of moderately high frequency (assuming
+the BYU data is to be trusted), it probably is a better claimant than either
+2of5core or 3esl to truly representing a core English vocabulary, at least of
+the American variety.
+
+Specialized 12 dicts lists
+
+The following table summarizes the contents of each of the lists in the Special
+directory, ordered by size in words:
+┌─────────────────┬────────┬────────┬───────┐
+│ │neol2016│2of5core│6phrase│
+├─────────────────┼────────┼────────┼───────┤
+│Size (Words) │600 │4,700 │22,000 │
+├─────────────────┼────────┼────────┼───────┤
+│Number of Sources│0 │5 │6 │
+├─────────────────┼────────┼────────┼───────┤
+│American English │Y │Y │Y │
+├─────────────────┼────────┼────────┼───────┤
+│British English │A little│Y │Y │
+├─────────────────┼────────┼────────┼───────┤
+│Ordinary words │Y │Y │– │
+├─────────────────┼────────┼────────┼───────┤
+│Inflections │Y │– │– │
+├─────────────────┼────────┼────────┼───────┤
+│Hyphenations │Y │A few │– │
+├─────────────────┼────────┼────────┼───────┤
+│Phrases │Y │A few │Y │
+├─────────────────┼────────┼────────┼───────┤
+│Names │Y │A few │A few │
+├─────────────────┼────────┼────────┼───────┤
+│Abbreviations │Y │A few │A few │
+├─────────────────┼────────┼────────┼───────┤
+│Acronyms │Y │A few │– │
+├─────────────────┼────────┼────────┼───────┤
+│Prefixes/Suffixes│– │– │– │
+├─────────────────┼────────┼────────┼───────┤
+│Signature words │– │– │* │
+├─────────────────┼────────┼────────┼───────┤
+│Neologisms │Y │– │– │
+├─────────────────┼────────┼────────┼───────┤
+│Annotations │Y │N │Y │
+└─────────────────┴────────┴────────┴───────┘
+
+A * in the "Signature Words" row means that signature words associated with
+some other list may be present, but there are no signature words associated
+specifically with that list.
+
+The neol2016 list
+
+The neol2016 list is a very simple list of new or newly recognized words, as
+described above. It is comprised of three parts, separated by blank lines.
+
+The first part lists regular (non-hyphenated, non-capitalized) words together
+with their inflections and variants, laid out similarly to the 2+2+3lem list.
+It includes plurals for uncountable nouns, marked with a "%" suffix. These
+words (except for the uncountable plurals) have been pre-added to the 2of12inf
+and 3of6game lists, suffixed with "!", allowing them to be easily removed if
+desired.
+
+The second part of the file is a small set of words for which additional
+inflections have been added. This portion of the file is in the same format as
+the first list. These inflections have also been added to the 2of12inf and
+3of6game lists.
+
+The third part of the file contains new words and phrases which are not regular
+words: hyphenated words, multi-word phrases, proper names, abbreviations and
+acronyms. These words have not been pre-added to any other list.
+
+In all cases, users are encouraged to add some or all of these words to any of
+the other lists, as they feel appropriate.
+
+The 2of5core list
+
+Five of the six advanced learner's ESL dictionaries from which the 3of6 lists
+were compiled mark a subset of their words as being important words which every
+student of English should master. These subsets vary widely from dictionary to
+dictionary. As one of the original goals of the 12dicts project was to compile
+a list representing the English core vocabulary, I thought it would be
+interesting to combine these lists. My original thought was to provide a list
+that was simply the union of the marked subsets for each source. However, one
+particular dictionary had at least twice as many words in its subset as any of
+the others, and in many cases the words seemed to me to be poorly chosen. (Do
+moor and cash flow seem like key English language concepts to you?) So I chose
+when assembling my list to require that all words be marked as important words
+by at least two of the sources. The result was the 2of5core list, which
+contains about 4,700 words.
+
+While most words selected in this way were the same in American and British
+English, some belonged to one variant or the other. In some cases, a word
+appeared in two forms, such as center and centre. When I observed that a word
+was present in two forms, I combined them into a single line, for example
+center/centre. No other changes were made to the list.
+
+Due to the way in which the list was constructed, it seems somewhat haphazard.
+You may want to check out the Oxford 3000™, a list of 3000 words available from
+Oxford University, which is a core vocabulary created by lexicographers, to my
+eye superior to the 2of5core list.
+
+The 6phrase list
+
+When I was compiling the 3of6all list, I noticed something interesting. There
+were an extraordinary number of phrases listed by only one of the sources. Many
+of these were extremely common phrases, which I would expect most experienced
+English speakers to understand. So, naturally, I decided to compile them all
+into a list.
+
+The 6phrase list contains all multi-word phrases from any of the six advanced
+learner's dictionaries which were used as sources for 3of6all, all 22,000 of
+them. The list does not include inflections, except in a few cases where a
+plural cannot easily be guessed from the words in a phrase. Usually, this
+happens for phrases of non-English origin, such as eau de cologne, whose plural
+is eaux de cologne. The list includes phrasal verbs, which are suffixed by the
+";" character, as in the 3of6all list. The list is sorted in a different order
+than the lexicographical ordering used by the other lists, in order to group
+all phrases starting with the same word together.
+
+You will observe that the same phrase will often be repeated several times in
+the list, with slightly different spelling, capitalization and/or hyphenation.
+No attempt was made to edit the list to remove or reduce such "clutter".
+
+The 6phrase list includes the 3of6all signature phrases. These are not marked
+with a suffix.
+
+In contrast to most of the other lists, I am unable to think of any
+applications of the 6phrase list. But I find it rather interesting, which is
+why I'm bothering to include it. At the very least, it may serve as an
+illustration of the incredible richness of the English language, without even
+venturing into vocabulary too esoteric to be included in a learner's
+dictionary.
How 12dicts came to be
-It may have occurred to some to wonder about how something like the
-n-dicts project came to be (though I assume that anyone who bothers to
-download this archive must already have some idea that such a project
-could be of interest).
-
-Some years ago, there was a post to the sci.crypt Usenet newsgroup, on
-the subject of creating PGP passphrases using randomly selected entries
-from a supplied list of very short words. (If this sounds interesting,
-follow this link for an expanded version of the post.) The word list,
-which was extracted from /usr/dict/words on some UNIX system, seemed to
-me ill-suited to its intended purpose. It included arcane acronyms (
-bstj, fmc), misspellings (diety, ouvre) and words of amazing obscurity (
-bhoy, kombu). I decided I could do better (and eventually did). This
-caused me to start downloading English word lists, of which there are
-many, from the Internet. I was not impressed by the overall quality of
-these lists, and the few which were high-quality were all-inclusive,
-burying the everyday words under a mountain of archaisms and esoterica.
-The flaws of the vast majority of these lists are worth recounting:
-
- * Failure to proofread. Many of these lists are littered with
- misspellings and typos, sometimes approaching gibberish. (I presume,
- for instance, that the bizarre string nondploe, which was found in a
- purported Scrabble word list, is a typo for something more or less
- legitimate, but I have no idea what.) Working on my own lists has
- helped me understand that 100 % accuracy is a very demanding goal,
- seldom actually achieved, but I still feel it reasonable to expect
- no more than 1 or 2 errors per 10,000 words.
- * Acceptance of completely undocumented lazy spellings, such as
- bullseye and courtmartial.
- * Failure to respect capitalization.
- * Failure to distinguish abbreviations from other entries.
- * Treating esoteric computer jargon, and especially UNIX jargon, as
- everyday English. (Beware any list which includes bitblt, emacs,
- inode or lvalue.)
- * Apparently random word selection. For instance, the most common
- version of /usr/dicts/words contains a large set of apparently
- randomly chosen personal names (uncapitalized, of course, and
- missing wanda, marge, polly and sid).
- * Inconsistent inflection. Some lists include all inflections of their
- vocabulary, while others include only singulars and infinitives.
- Either policy is fine, and has its advantages. I am personally very
- annoyed when inflected forms appear at random. I find this generally
- happens when a compiler merges several lists with different
- characteristics, with no attempt to reconcile their divergent
- styles.
- * Omission of everyday words. I've seen a purported general-purpose
- list that includes bremsstrahlung, yet omits log and beer. Or that
- includes saxophone but not sax, and rhinoceros but not rhino. Of
- course, due to my original purpose in seeking out common short
- words, I found this especially annoying.
-
-One result of my frustration with this situation was my working with
-Mendel Cooper on ENABLE (for further information, check out this link),
-which was close to unique in having an active caretaker, one clearly
-concerned with quality, and in being oriented towards American rather
-than British English. But ENABLE is an all-encompassing list and, even
-if it had been complete at the time I started my search for a list of
-common words, it would not have been what I wanted for that reason.
-
-I finally decided that only starting from scratch with a systematic
-approach was likely to get me what I was looking for, and that
-dictionaries intended for non-native speakers of English were the best
-possible source for words that are in some cases so familiar that we
-never think of them. This has led to the 12dicts lists, which I hope
-have managed to avoid the flaws recited above.
-
-(I should acknowledge one form of inconsistency exhibited by the 12dicts
-lists, which is that sometimes related words are spelled inconsistently.
-For instance, the 2of12 list contains both broadminded and
-broad-mindedness. This generally occurs as a result of the methodology
-used to build the lists. In the case of broadminded, only one source
-dictionary listed broadmindedness, which was therefore excluded. I felt
-unequal to trying to correct these inconsistencies, some of which are
-real and not mere artifacts of 12dicts, such as the contrast between
-self-conscious and unselfconscious.)
+It may have occurred to some to wonder about how something like the 12dicts
+project came to be (though I assume that anyone who bothers to download this
+archive must already have some idea that such a project could be of interest).
+
+Many years ago, there was a post to the sci.crypt Usenet newsgroup, on the
+subject of creating PGP passphrases using randomly selected entries from a
+supplied list of very short words. (If this sounds interesting, follow this
+link for an expanded version of the post.) The word list, which was extracted
+from /usr/dict/words on some UNIX system, seemed to me ill-suited to its
+intended purpose. It included arcane acronyms (bstj, fmc), misspellings (diety,
+ouvre) and words of amazing obscurity (bhoy, kombu). I decided I could do
+better, and eventually did. This caused me to start downloading English word
+lists, of which there were many, from the Internet. I was not impressed by the
+overall quality of these lists, and the few which were high-quality were
+all-inclusive, burying the everyday words under a mountain of archaisms and
+esoterica.
+
+This was a long time ago, and an Internet search for word lists now turns up
+lists of higher quality than back then (thanks in part to the influence of
+12dicts), so I will limit myself to two brief criticisms of the various lists
+available at that time. First, they contained far too many misspellings and
+typos, and had obviously never been proofread. Additionally, their approach to
+vocabulary was scattershot, omitting common words while adding a random
+selection of highly technical words, often associated with UNIX and academic
+computer science. (My favorite is the list which included bremsstrahlung, but
+omitted log and beer.) Due to my original purpose of finding a list of short,
+common words, I found this sort of thing particularly frustrating.
+
+One result of my frustration with this situation was my working with Mendel
+Cooper on ENABLE, a large Scrabble®-oriented list, which was close to unique in
+having an active caretaker who was clearly concerned with quality, and in being
+oriented towards American rather than British English. But ENABLE was an
+all-encompassing list and, even if it had been complete at the time I started
+my search for a list of common words, it would not have been what I wanted for
+that reason. (The ENABLE web site is no longer online, but a Google search will
+turn up places where you can still download it.)
+
+I finally decided that only starting from scratch with a systematic approach
+was likely to get me what I was looking for, and that dictionaries intended for
+non-native speakers of English were the best possible source for words that are
+in some cases so familiar that we never think of them. This has led to the
+12dicts lists, which I hope have managed to avoid the flaws recited above.
+
+My other projects
+
+During the intervals between releases of 12dicts, I have been fooling around
+with English spelling reform. One of the results of this activity is the
+development of CAAPR and ABCD, both of which may be downloaded from my website,
+www.wyrdplay.org. CAAPR is the Combined Anglo-American Pronunciation Reference,
+a fancy name for a bi-dialectal pronunciation dictionary whose word list is
+derived primarily from the 12dicts 6of12 list. ABCD, Alan's Basic Codes with
+Diacritics, is also a pronunciation dictionary, of a somewhat different sort -
+the notation is designed to clarify when a word is spelled in accordance with
+normal English spelling patterns (as with fault or tunnel), and when it is not
+(as with fought or colonel). Though these files were developed as a result of
+my interest in spelling reform, they may be of interest to other "word nerds"
+unconcerned with that particular quixotic pastime.
+
+Click the following links to CAAPR and ABCD if interested.
Conclusions
-When I released the first version of 12dicts in 1999, I assumed I was
-done with it. It hasn't worked out that way. Before I declare it
-finished for a second time, there are a few more things I'd like to
-accomplish.
-
- * As mentioned above, I would like to rework the 2of12inf list to
- remove the dependency on the Moby lists.
- * As may be seen by inspecting the table of file characteristics, the
- 12dicts files now form a spectrum of word lists, with contents
- ranging from the extremely common to the mildly esoteric. I would
- like to extend the spectrum further by applying the 12dicts
- methodology to dictionaries of larger size. Whether I will ever get
- the time for a project this large remains to be seen. If it ever
- comes to pass, it will probably be released separately from 12dicts
- itself, as anything larger than the 5desk list will be too large to
- even pretend to represent a "core English" vocabulary. (Even the
- 5desk list itself is too large for that purpose.)
- * It is possible that in the future the "n" of n-dicts will increase
- again, but, in fact, consideration of an additional dictionary now
- generally ends with the discovery that its vocabulary matches
- 12dicts pretty closely. At the very least, this phenomenon gives me
- hope that the 12dicts lists have now fulfilled their basic purpose.
-
-The 12dicts lists were compiled by Alan Beale. I explicitly release them
-to the public domain, but request acknowledgment of their use.
-(Actually, the dependency of the 2of12inf list on AGID prevents its
+When I released the first version of 12dicts in 1999, I assumed I was done with
+it. It hasn't worked out that way. I now think I'm pretty much done with it
+again, though an occasional update to neol20xx.txt might be called for. Perhaps
+in ten more years I'll have reached version 9, and be laughing uncontrollably
+at the thought that I might have finished earlier, but for the present I don't
+see what else might be both useful and fun to add.
+
+Feel free to send comments, suggestions, inquiries and/or large sums of money
+to me at 12dicts@pobox.com. (Actually, the bit about money is a joke. Do not
+send me even small amounts of money; 12dicts is free wordware.) After making
+this request in previous versions, I have been delighted to see the interest in
+these lists for projects ranging from interactive games to literacy programs.
+And I have been particularly pleased to occasionally hear of first-year
+Computer Science assignments specifying a 12dicts list rather than /usr/dict/
+words for their input. Keep up the good work, and do let me know what you're
+doing. (Oh, and please put "12dicts" in the subject line when you email me.
+This will allow me to easily notice your mail even if it is misclassified by an
+overzealous filter as spam. Speaking of spam, the publication of my email
+address in this package has led to a marked increase in the amount of spam I
+receive and, ironically, much of it contains subject lines which appear to have
+been extracted at random from my own lists. This is a use of 12dicts of which I
+do not approve!)
+
+The 12dicts lists were compiled by Alan Beale. I explicitly release them to the
+public domain, but request acknowledgment of their use. (Actually, the
+dependency of the 2of12inf list and the 2+2+3 lists on AGID prevents their
release into the public domain. However, I do not impose any additional
-requirements on its use beyond those imposed by AGID and its sources, as
-described in agid.txt.) Feel free to send comments, suggestions,
-inquiries and/or large sums of money to me at biljir@pobox.com. If you
-find 12dicts useful, I'd love to hear about it.
+requirements on their use beyond those imposed by AGID and its sources, as
+described in agid.txt.)
+
+- Alan Beale -
projects / deb_pkgs / scowl.git / commitdiff