More on words, words, words ...

As we've seen, the English language has certain common words that all writers must use and they use them often. We expect them, and our subconscious looks for them as we learn to read.

I selected another arbitrary (and brief) short story, Arthur Conan Doyle's "Horror of the Heights". The top twenty words for Doyle are: (394, the); (223, of); (209, and); (181, I); (177, a); (142, to); (140, was); (122, it); (113, in); (98, That); (94, my); (71, as); (56, with); (51, but); (49, which); (48, is); (47, me); (45, had); (44, at); (43, for). These are virtually identical to Lovecraft's story, and would be so for most any other work of fiction.

It's not necessarily the use of common words, but a writer's choices of uncommon words, and the selective exclusion (if any) of common words we notice.

Statistically, Doyle uses only one instance of a word [that is the single use of one word rather than the use multiple times] 16.9% of the time in this story (1138 of 6779 words). This is far less than Lovecraft's Dagon 29.7% (661 unique words out of 2239 words).

Obviously much work would have to be done, but it seems that Lovecraft did not repeat himself often except for effect. This may give rise to the impression that Lovecraft's fiction was erudite, especially as he was quickly known for use of archaic words, technical terms, ad hoc made up words, and unusual adjectives or adverbs (like cyclopean).