1 Spell Checking Oriented Word Lists (SCOWL)
6 The SCOWL is a collection of word lists split up in various sizes, and
7 other categories, intended to be suitable for use in spell checkers.
8 However, I am sure it will have numerous other uses as well.
10 The latest version can be found at http://wordlist.sourceforge.net/
12 The directory final/ contains the actual word lists broken up into
13 various sizes and categories. The r/ directory contains Readmes from
14 the various sources used to create this package.
16 The other directories contain the necessary information to recreate the
17 word lists from the raw data. Unless you are interested in improving the
18 words lists you should not need to worry about what's here. See the
19 section on recreating the words lists for more information on what's
22 Except for the special word lists the files follow the following
24 <spelling category>-<classification>.<size>
25 Where the spelling category is one of
26 english, american, british, british_z, canadian,
27 variant_0, varaint_1, variant_2
28 Classification is one of
29 abbreviations, contractions, proper-names, upper, words
31 10, 20, 35 (small), 40, 50 (medium), 55, 60, 70 (large),
32 80 (huge), 95 (insane)
33 The special word lists follow are in the following format:
34 special-<description>.<size>
35 Where description is one of:
36 roman-numerals, hacker
38 When combining the words lists the "english" spelling category should
39 be used as well as one of "american", "british", "british_z" (british
40 with ize spelling), or "canadian". Great care has been taken so that
41 that only one spelling for any particular word is included in the main
42 list. When two variants were considered equal I randomly picked one
43 for inclusion in the main word list. Unfortunately this means that my
44 choice in how to spell a word may not match your choice. If this is
45 the case you can try including the "variant_0" spelling category which
46 includes most variants which are considered almost equal. The
47 "variant_1" spelling category include variants which are also
48 generally considered acceptable, and "variant_2" contains variants
49 which are seldom used.
51 The "abbreviation" category includes abbreviations and acronyms which
52 are not also normal words. The "contractions" category should be self
53 explanatory. The "upper" category includes upper case words and proper
54 names which are common enough to appear in a typical dictionary. The
55 "proper-names" category included all the additional uppercase words.
56 Final the "words" category contains all the normal English words.
58 To give you an idea of what the words in the various sizes look like
59 here is a sample of 25 random words found only in that size:
61 10: began both buffer cause collection content documenting easiest
62 equally examines expecting first firstly hence inclining
63 irrelevant justified little logs necessarily ought sadly six
66 20: chunks commodity contempt contexts cruelty crush dictatorship
67 disgusted dose elementary evolved frog god hordes notion overdraft
68 overlong overlook phoning poster recordings sand skull substituted
71 35: aliasing blackouts blowout bluntness corroborated derrick
72 dredging elopements entrancing excising fellowship flagpole
73 germination glimpse gondola guidebook madams minimalism minnows
74 partisans petitions shelling swarmed throng welding
76 40: altercation blender castigation chump coffeehouse determiners
77 doggoning exhibitor finders flophouse gazebo lumbering masochism
78 mopeds poetically pubic refinance reggae scragglier softhearted
79 stubbornness teargassed township underclassman whoosh
81 50: accumulative adulterant allegorically amorousness astrophysics
82 camphor coif dickey elusiveness enviousness fakers fetishistic
83 flippantly headsets liefs midyears myna pacification persiflage
84 phosphoric pinhole sappy seres unrealistically unworldly
86 55: becquerel brickie centralist cine conveyancing courgette
87 disarmingly garçon gobstopper infilling insipidity
88 internationalist kabuki lyrebirds obscurantism rejigged
89 revisionist satsuma slapper sozzled sublieutenants teletext vino
92 60: absorber acceptableness adventurousness antifascists arrhythmia
93 audiology cartage cruses fontanel forelimbs granter hairlike
94 installers jugglery lappets libbers mandrels micrometeorite
95 mineshaft reconsecrates saccharides smellable spavined sud timbrel
97 70: atomisms benedict carven coxa cyanite detraining diazonium
98 dogberry dogmatics entresol fatherlessnesses firestone imprecator
99 laterality legitimisms maxwell microfloppies nonteaching pelerine
100 pentane pestiferousness piscator profascist tusche twirp
102 80: cotransfers embrangled forkednesses giftwrapped globosity hatpegs
103 hepsters hermitess interspecific inurbanities lamiae
104 literaehumaniores literatures masulas misbegun plook prerupt
105 quaalude rosanilin sabbatism scowder subreptive thumbstalls
108 95: anatropal anientise bakshi brouzes corsie daimiote dhaw dislikened
109 ectoretina fortuitisms guardeen hyperlithuria nonanachronistic
110 overacceleration pamphletic parma phytolith starvedly
111 trophoplasmic ulorrhagia undared undertide unplunderously
112 unworkmanly vasoepididymostomy
114 And here is a rough count on the number of words in the "english"
115 spelling category for each size:
117 Size Words Proper Names Running Total
123 50 23,200 17,200 95,000
125 60 16,000 12,800 132,000
126 70 45,100 34,300 211,000
127 80 137,000 30,400 379,000
128 95 198,000 51,800 628,000
130 (The "Words" column does not include the proper name count.)
132 Size 35 is the recommended small size, 50 the medium and 70 the large.
133 Sizes 70 and below contain words found in most dictionaries while the
134 80 size contains all the strange and unusual words people like to use
135 in word games such as Scrabble (TM). While a lot of the the words in
136 the 80 size are not used very often, they are all generally considered
137 valid words in the English language. The 95 contains just about every
138 English word in existence and then some. Many of the words at the 95
139 level will probally not be considered valid english words by most
140 people. I don't recommend anyone use levels above 70 for spell
141 checking as they contain rarely used words which can hide misspellings
142 of similar more commonly used words. For example the word "ort" can
143 hide a common typo of "or". No one should need to use a size larger
144 than 80, the 95 size is labeled insane for a reason.
146 Accents are present on certain words such as café in iso8859-1 format.
150 From Revision 5 to 6 (August 10, 2004)
152 Updated to version 4.0 of the 12dicts package.
154 Included the 3esl, 2of4brif, and 5desk list from the new 12dicts
155 package. The 3esl was included in the 40 size, the 2of4brif in the
156 55 size and the 5desk in the 70 size.
158 Removed the Ispell word list as it was a source of too many errors.
159 This eliminated the 65 size.
161 Removed clause 4 from the Ispell copyright with permission of Geoff
164 Updated to version 4.1 of VarCon.
166 Added the "british_z" spelling category which it British using the
169 From Revision 4a to 5 (January 3, 2002)
171 Added variants that were not really spelling variants (such as
172 forwards) back into the main list.
174 Fixed a bug which caused variants of words to incorrectly appear in
175 the non-variant lists.
177 Moved rarly used inflections of a word into higher number lists.
179 Added other inflections of a words based on the following criteria
180 If the word is in the base form: only include that word.
181 If the word is in a plural form: include the base word and the plural
182 If the word is a verb form (other than plural): include all verb forms
183 If the word is an ad* form: include all ad* forms
184 If the word is in a possessive form: also include the non-possessive
186 Updated to the latest version of many of the source dictionaries.
188 Removed the DEC Word List due to the questionable licence and
189 because removing it will not seriously decrese the quality of SCOWL
190 (there are a few less proper names).
192 From Revision 4 to 4a (April 4, 2001)
194 Reran the scripts on a never version of AGID (3a) which fixes a bug
195 which caused some common words to be improperly marked as variants.
197 From Revision 3 to 4 (January 28, 2001)
199 Split the variant "spelling category" up into 3 different levels.
201 Added words in the Ispell word list at the 65 level.
203 Other changes due to using more recent versions of various sources
204 included a more accurete version of AGID thanks to the word of
207 From Revision 2 to 3 (August 18, 2000)
209 Renamed special-unix-terms to special-hacker and added a large
210 number of communly used words within the hacker (not cracker)
213 Added a couple more signature words including "newbie".
215 Minor changes due to changes in the inflection database.
217 From Revision 1 to 2 (August 5, 2000)
219 Moved the male and female name lists from the mwords package and the
220 DEC name lists form the 50 level to the 60 level and moved Alan's
221 name list from the 60 level to the 50 level. Also added the top
222 1000 male, female, and last names from the 1990 Census report to the
223 50 level. This reduced the number of names in the 50 level from
226 Added a large number of Uppercase words to the 50 level.
228 Properly accented the possessive form of some words.
230 Minor other changes due to changes in my raw data files which have
231 not been released yet. Email if you are interested in these files.
233 COPYRIGHT, SOURCES, and CREDITS:
235 The collective work is Copyright 2000-2004 by Kevin Atkinson as well
236 as any of the copyrights mentioned below:
238 Copyright 2000-2004 by Kevin Atkinson
240 Permission to use, copy, modify, distribute and sell these word
241 lists, the associated scripts, the output created from the scripts,
242 and its documentation for any purpose is hereby granted without fee,
243 provided that the above copyright notice appears in all copies and
244 that both that copyright notice and this permission notice appear in
245 supporting documentation. Kevin Atkinson makes no representations
246 about the suitability of this array for any purpose. It is provided
247 "as is" without express or implied warranty.
249 Alan Beale <biljir@pobox.com> also deserves special credit as he has,
250 in addition to providing the 12Dicts package and being a major
251 contributor to the ENABLE word list, given me an incredible amount of
252 feedback and created a number of special lists (those found in the
253 Supplement) in order to help improve the overall quality of SCOWL.
255 The 10 level includes the 1000 most common English words (according to
256 the Moby (TM) Words II [MWords] package), a subset of the 1000 most
257 common words on the Internet (again, according to Moby Words II), and
258 frequently class 16 from Brian Kelk's "UK English Wordlist
259 with Frequency Classification".
261 The MWords package was explicitly placed in the public domain:
263 The Moby lexicon project is complete and has
264 been place into the public domain. Use, sell,
265 rework, excerpt and use in any way on any platform.
267 Placing this material on internal or public servers is
268 also encouraged. The compiler is not aware of any
269 export restrictions so freely distribute world-wide.
271 You can verify the public domain status by contacting
275 Arcata, CA 95521-4884
280 The "UK English Wordlist With Frequency Classification" is also in the
283 Date: Sat, 08 Jul 2000 20:27:21 +0100
284 From: Brian Kelk <Brian.Kelk@cl.cam.ac.uk>
286 > I was wondering what the copyright status of your "UK English
287 > Wordlist With Frequency Classification" word list as it seems to
288 > be lacking any copyright notice.
290 There were many many sources in total, but any text marked
291 "copyright" was avoided. Locally-written documentation was one
292 source. An earlier version of the list resided in a filespace called
293 PUBLIC on the University mainframe, because it was considered public
296 Date: Tue, 11 Jul 2000 19:31:34 +0100
298 > So are you saying your word list is also in the public domain?
300 That is the intention.
302 The 20 level includes frequency classes 7-15 from Brian's word list.
304 The 35 level includes frequency classes 2-6 and words appearing in at
305 least 11 of 12 dictionaries as indicated in the 12Dicts package. All
306 words from the 12Dicts package have had likely inflections added via
307 my inflection database.
309 The 12Dicts package and Supplement is in the Public Domain.
311 The WordNet database, which was used in the creation of the
312 Inflections database, is under the following copyright:
314 This software and database is being provided to you, the LICENSEE,
315 by Princeton University under the following license. By obtaining,
316 using and/or copying this software and database, you agree that you
317 have read, understood, and will comply with these terms and
320 Permission to use, copy, modify and distribute this software and
321 database and its documentation for any purpose and without fee or
322 royalty is hereby granted, provided that you agree to comply with
323 the following copyright notice and statements, including the
324 disclaimer, and that the same appear on ALL copies of the software,
325 database and documentation, including modifications that you make
326 for internal use or for distribution.
328 WordNet 1.6 Copyright 1997 by Princeton University. All rights
331 THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
332 UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
333 IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
334 UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
335 ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE
336 LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT INFRINGE ANY
337 THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
339 The name of Princeton University or Princeton may not be used in
340 advertising or publicity pertaining to distribution of the software
341 and/or database. Title to copyright in this software, database and
342 any associated documentation shall at all times remain with
343 Princeton University and LICENSEE agrees to preserve same.
345 The 40 level includes words from Alan's 3esl list found in version 4.0
346 of his 12dicts package. Like his other stuff the 3esl list is also in the
349 The 50 level includes Brian's frequency class 1, words words appearing
350 in at least 5 of 12 of the dictionaries as indicated in the 12Dicts
351 package, and uppercase words in at least 4 of the previous 12
352 dictionaries. A decent number of proper names is also included: The
353 top 1000 male, female, and Last names from the 1990 Census report; a
354 list of names sent to me by Alan Beale; and a few names that I added
355 myself. Finally a small list of abbreviations not commonly found in
356 other word lists is included.
358 The name files form the Census report is a government document which I
359 don't think can be copyrighted.
361 The file special-jargon.50 uses common.lst and word.lst from the
362 "Unofficial Jargon File Word Lists" which is derived from "The Jargon
363 File". All of which is in the Public Domain. This file also contain
364 a few extra UNIX terms which are found in the file "unix-terms" in the
367 The 55 level includes words from Alan's 2of4brif list found in version
368 4.0 of his 12dicts package. Like his other stuff the 2of4brif is also
369 in the public domain.
371 The 60 level includes Brian's frequency class 0 and all words
372 appearing in at least 2 of the 12 dictionaries as indicated by the
373 12Dicts package. A large number of names are also included: The 4,946
374 female names and the 3,897 male names from the MWords package.
376 The 70 level includes the 74,550 common dictionary words and the
377 21,986 names list from the MWords package The common dictionary words,
378 like those from the 12Dicts package, have had all likely inflections
379 added. The 70 level also included the 5desk list from version 4.0 of
380 the 12Dics package which is the public domain
382 The 80 level includes the ENABLE word list, all the lists in the
383 ENABLE supplement package (except for ABLE), the "UK Advanced Cryptics
384 Dictionary" (UKACD), the list of signature words in from YAWL package,
385 and the 10,196 places list from the MWords package.
387 The ENABLE package, mainted by M\Cooper <thegrendel@theriver.com>,
388 is in the Public Domain:
390 The ENABLE master word list, WORD.LST, is herewith formally released
391 into the Public Domain. Anyone is free to use it or distribute it in
392 any manner they see fit. No fee or registration is required for its
393 use nor are "contributions" solicited (if you feel you absolutely
394 must contribute something for your own peace of mind, the authors of
395 the ENABLE list ask that you make a donation on their behalf to your
396 favorite charity). This word list is our gift to the Scrabble
397 community, as an alternate to "official" word lists. Game designers
398 may feel free to incorporate the WORD.LST into their games. Please
399 mention the source and credit us as originators of the list. Note
400 that if you, as a game designer, use the WORD.LST in your product,
401 you may still copyright and protect your product, but you may *not*
402 legally copyright or in any way restrict redistribution of the
403 WORD.LST portion of your product. This *may* under law restrict your
404 rights to restrict your users' rights, but that is only fair.
406 UKACD, by J Ross Beresford <ross@bryson.demon.co.uk>, is under the
409 Copyright (c) J Ross Beresford 1993-1999. All Rights Reserved.
411 The following restriction is placed on the use of this publication:
412 if The UK Advanced Cryptics Dictionary is used in a software package
413 or redistributed in any form, the copyright notice must be
414 prominently displayed and the text of this document must be included
417 There are no other restrictions: I would like to see the list
418 distributed as widely as possible.
420 The 95 level includes the 354,984 single words and 256,772 compound
421 words from the MWords package, ABLE.LST from the ENABLE Supplement,
422 and some additional words found in my part-of-speech database that
423 were not found anywhere else.
425 Accent information was taken from UKACD.
427 My VARCON package was used to create the American, British, and
430 Since the original word lists used used in the VARCON package came
431 from the Ispell distribution they are under the Ispell copyright:
433 Copyright 1993, Geoff Kuenning, Granada Hills, CA
436 Redistribution and use in source and binary forms, with or without
437 modification, are permitted provided that the following conditions
440 1. Redistributions of source code must retain the above copyright
441 notice, this list of conditions and the following disclaimer.
442 2. Redistributions in binary form must reproduce the above copyright
443 notice, this list of conditions and the following disclaimer in the
444 documentation and/or other materials provided with the distribution.
445 3. All modifications to the source code must be clearly marked as
446 such. Binary redistributions based on modified source code
447 must be clearly marked as modified versions in the documentation
448 and/or other materials provided with the distribution.
449 (clause 4 removed with permission from Geoff Kuenning)
450 5. The name of Geoff Kuenning may not be used to endorse or promote
451 products derived from this software without specific prior
454 THIS SOFTWARE IS PROVIDED BY GEOFF KUENNING AND CONTRIBUTORS ``AS
455 IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
456 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
457 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL GEOFF
458 KUENNING OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
459 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
460 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
461 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
462 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
463 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
464 ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
465 POSSIBILITY OF SUCH DAMAGE.
467 The variant word lists were created from a list of variants found in
468 the 12dicts supplement package as well as a list of variants I created
471 The Readmes for the various packages used can be found in the
472 appropriate directory under the r/ directory.
476 There is a very nice frequency analyse of the BNC corpus done by
477 Adam Kilgarriff. Unlike Brain's word lists the BNC lists include part
478 of speech information. I plan on somehow using these lists as Adam
479 Kilgarriff has given me the OK to use it in SCOWL. These lists will
480 greatly reduce the problem of inflected forms of a word appearing at
481 different levels due to the part-of-speech information.
483 I also plan on perhaps putting the data in a database and use SQL
484 queries to create the wordlists instead of tons of "sort"s, "comm"s,
487 RECREATING THE WORD LISTS:
489 In order to recreate the word lists you need a modern version of Perl,
490 bash, the traditional set of shell utilities, a system that supports
491 symbolic links, and quite possibly GNU Make. Once you have downloaded
492 all the necessary raw data in the r/ directory you should be able to
493 type "rm final/* && make all" and the word lists in the final/
494 directory should be recreated. If you have any problems fell free to
495 contact me; however, unless you are interested in improving the
496 scripts used, I will likely ignore you as there should be little need
497 for anyone not interested in improving the word list to do so.
499 The src/ directory contains the numerous scripts used in the creation
500 of the final product.
502 The r/ directory contains the raw data used to
503 create the final product. In order for the scripts to work various
504 word lists and databases need to be created and put into this
505 directory. See the README file in the r/ directory for more
508 The l/ directory contains symbolic links used by the actual scripts.
510 Finally, the working/ directory is where all the intermittent files go
511 that are not specific to one source.