linguist | Language Savant. If your repository's language is being reported incorrectly, send us a pull request

something like this?

The short answer is that BASIC numbering is in increments of one, but programmers can and do skip some of the increments. BASIC grew out of Fortran, which also used numeric labels, and often used increments of 10. Unlike Fortran, early BASIC required numbering all lines, so that they changed from labels to line numbers.

BASIC is numbered in increments greater than one to allow adding new lines between existing lines.

1. Most early home computer BASIC implementations did not have a built-in means of renumbering lines.
2. Code execution in BASIC implementations with line numbers happened in order of line number.

This meant that if you wanted to add new lines, you needed to leave numbers free between those lines. Even on computers with a RENUM implementation, renumbering could take time. So if you wanted standard increments you’d still usually only RENUM at the end of a session or when you thought you were mostly finished.

Speculation: Programmers use increments of 10 specifically for BASIC line numbers for at least two reasons. First, tradition. Fortran code from the era appears to use increments of 10 for its labels when it uses any standard increments at all. Second, appearance. On the smaller screens of the era it is easier to see where BASIC lines start if they all end in the same symbol, and zero is a very useful symbol for that purpose. Speaking from personal experience, I followed the spotty tradition of starting different routines on hundreds boundaries and thousands boundaries to take advantage of the multiple zeroes at the beginning of the line. This made it easier to recognize the starts of those routines later when reading through the code.

BASIC grew from Fortran, which also used numbers, but as labels. Fortran lines only required a label if they needed to be referred to, such as with a GO TO, to know where a loop can be exited, or as a FORMAT for a WRITE. Such lines were also often in increments greater than 1—and commonly also 10—so as to allow space to add more in between if necessary. This wasn’t technically necessary. Since they were labels and not line numbers, they didn’t need to be sequential. But most programmers made them sequential for readability.

In his commonly-used Fortran 77 tutorial, Erik Boman writes:

Typically, there will be many loops and other statements in a single program that require a statement label. The programmer is responsible for assigning a unique number to each label in each program (or subprogram). The numerical value of statement labels have no significance, so any integer numbers can be used. Typically, most programmers increment labels by 10 at a time.

BASIC required that all lines have numbers and that the line numbers be sequential; that was part of the purpose of having line numbers: a BASIC program could be entered out of order. This allowed for later edits. Thus, line 15 could be added after lines 10 and 20 had been added. This made leaving potential line numbers between existing line numbers even more useful.

If you look at magazines with BASIC program listings, such as Rainbow Magazine or Creative Computing, you’ll often see numbers sandwiched somewhat randomly between the tens. And depending on style, many people used one less than the line number at the start of a routine or subroutine to comment the routine. Routines and DATA sections might also start on even hundreds or even thousands.

Programmers who used conventions like this might not even want to renumber a program, as it would mess up their conventions. BASIC programs were often a mass of text; any convention that improved readability was savored.

Ten was a generally accepted spacing even before the home computer era. In his basic basic, second edition (1978, and expecting that the user would be using “a remote terminal”), James S. Coan writes (page 2):

It is conventional although not required to use intervals of 10 for the numbers of adjacent lines in a program. This is because any modification in the program must also have line numbers. So you can use the in-between numbers for that purpose. It should be comforting to know at this point that the line numbers do not have to be typed in order. No matter what order they are typed in, the computer will follow the numerical order in executing the program.

There are examples of similar patterns in Coan’s Basic Fortran. For example, page 46 has a simple program to “search for pythagorean triples”; while the first label is 12, the remaining labels are 20, 30, and 40, respectively.

He used similar patterns without increments of 10; for example, on page 132 of Basic Fortran, Coan uses increments of 2 for his labels, and keeps the calculation section of the program in the hundreds with the display section of the program in the two hundreds. The END statement uses label 9900.

Similarly, in their 1982 Elementary BASIC, Henry Ledgard and Andrew Singer write (page 27):

Depending on the version of Basic you are using, a line number can consist of 1 to 4 or 5 digits. Here, all line numbers will consist of 4 digits, a common practice accepted by almost every version of Basic. The line numbers must be in sequential order. Increasing line numbers are often given in increments of 10, a convention we will also follow. This convention allows you to make small changes to a program without changing all the line numbers.

And Jerald R. Brown’s 1982 Instant BASIC: 2nd Astounding Edition (p. 7):

You don’t have to enter or type in a program in line number order. That is, you don’t have to enter line 10 first, then line 20, and then line 30. If we type in a program out of line number order, the computer doesn’t care. It follows the line numbers not the order they were entered or typed in. This makes it easy to insert more statements in a program already stored in the computer’s memory. You may have noticed how we cleverly number the statements in our programs by 10's. This makes it easy to add more statements between the existing line numbers -- up to nine more statements between lines 10 and 20, for example.

Much of the choice of how to number lines in a BASIC program was based on tradition and a vague sense of what worked. This was especially true in the home computer era where most users didn’t take classes on how to use BASIC but rather learned by reading other people’s programs, typing them in from the many books and magazines that provided program listings. The tradition of incrementing by 10 and inserting new features between those increments was an obvious one.

You can see it scanning through old books of code, such as 101 BASIC Computer Games. The very first program, “Amazin” increments its line numbers by 10. But at some point, a user/coder decided they needed an extra space after the code prints out how many dollars the player has; so that extra naked PRINT is on line 195. And the display of the instructions for the game are all kept between lines 100 and 109, another common pattern.

The program listing on page 30 for Basket displays the common habit of starting separate routines at even hundreds and thousands. Line numbers within those routines continue to increment by 10. The pattern is fairly obvious even though new features (and possibly other patterns) have added several lines outside the pattern.

As BASIC implementations began to get RENUM commands, more BASIC code listings appeared with increments of one. This is partly because using an increment of one used less memory. While the line number itself used a fixed amount of RAM (with the result that the maximum line number was often somewhere around FFFF, or 65525), references to line numbers did not tend to use a fixed length. Thus, smaller line numbers used less RAM overall.

Depending on how large the program was, and how much branching it used, this could be significant compared to the amount of RAM the machine itself had.

For example, I recently typed in the SKETCH.BAS program from the October 1984 Rainbow Magazine, page 97. This is a magazine, and a program, for the TRS-80 Color Computer. This program uses increments of 1 for its line numbering. On CLOADing the program in, free memory stands at 17049. After using RENUM 10,1,10 to renumber it in increments of 10, free memory stands at 16,953.

A savings of 96 bytes may not sound like much, but this is a very small program; and it’s still half a percent of available RAM. The difference could be the difference between a program fitting into available RAM or not fitting. This computer only has 22823 bytes of RAM free even with no program in memory at all.

To access the name of these items, just do function.name(). You could use line comprehension update these items as follows:

Looks like the else branch of if 'description' in dic: is never executed. If your code is indented correctly, then also the csvWriter.writerow part is never executed because of this.

That yields that no contents are written to your file.

A comment on code style:

• use with open(file) as file_variable: instead of manually using open and close. That can save you some trouble, e.g. the trouble you would get when the else branch would indeed be executed and the file would be closed multiple times :)

"Persisting" in most cases means "storing". Just that. Fancy word for simple thing.

It doesn't really matter if it's database storage, file storage, internet cloud storage, or, yes, even in-memory storage -- even though "in-memory" doesn't usually coincide with thinking about storage, since it tends to evaporate when the power goes out. It's more about how you are going to use this data, rather than where you are going to store it. So even building a in-memory most-recently-used list of terms can be thought as "persisting", if it's kept long enough in a typical non-disastrous case.

So, yes, storing unicode strings in a database is exactly as good use case for the word "persisting" as it can be.

--

However, for your use-case of determining whever the text entered by the user has changed, I'm not sure if you should focus on the 'persisting' aspect. At this point I think you don't really care about storing, you care about "has it been changed", and that should determine the choice of string comparison.

All StringComparison flags have some effects. Culture vs Invariant, Case-Sensitive or not, how would you like the comparison to behave, so that the result will be clear and understandable to the user?

If old text was "Mary has a lamb" and new text is "mAry HaS a Lamb", should it be treated as a change?

If old text was "Maria hat ein weißes Lamm" and new text is "Maria hat ein weisses Lamm", should it be treated as a change?

and so on. Right now I can't think of better examples, I'm sure you could think of some when focusing on your userbase and on what will be of the most use to them.

Please note, that they also may not care, and that you may be overthinking it or focusing on too tiny details too soon. Maybe the default comparison would be just fine for first few years until users tell you it could be better here and there? Dunno. YMMV :)

You can't put sentence boundaries at arbitrary characters - spaCy won't let you put a sentence in the middle of a token.

What you can do is iterate over tokens and use token.idx (the character index of the token) to apply your boundaries by finding the token that lines up with your boundary index. You'll have to figure out a policy for what to do if token boundaries don't line up with your values, whether that's throwing an exception or dealing with it somehow.

This is, in the general case, impossible. The halting problem precludes any program from being able to compute, in finite time, what other programs may or may not do - including what dependencies it requires to run. Sure, you can make it work for some inputs - but never for all.

So you have to compromise:

• which languages do you need to support? it-depends does not try to support Java, for example. Different languages have different ways of calling in dependencies from their source-code. For example, if working with C, you will want to look at #includes.
• which build-chains to you need to support? parsing a standard Makefile for C is very different from, say, looking into a Maven pom.xml for Java. Additionally, build-chains can perform arbitrary computation -- and again, due to the halting problem, your dependency-detection program will not be able to "statically" figure out intended behavior. It is entirely possible to link against one library or another one (or none at all) depending on what is detected to exist. What should you output in this case?. For programs that have no documented build process, you simply cannot know their dependencies. Often, the build-process is human-documented but not machine-readable...
• what do you consider a library/framework? long-lived libraries can evolve through many different versions, and the fact that one version is required and not another may not be explicit in the source-code. If a code-base depends on behavior found in only a specific, now superseded, version of a library, and no explicit mention of that version is found -- your dependency-detection program will have no way to know about it (unless you code in library-version-specific detection; which is doable, but on a case-by-case basis, and requires deep knowledge of differences between versions).

Therefore the answer to your question is that... it depends (they go into a fair amount of detail regarding limitations). For the specific case of Java + Maven, which is not covered by it-depends, you can use Maven itself, via mvn dependency:tree. Choose a subset of the problem instead of trying to solve it all at once.

Look for the "label scheme" on the page for any individual language.

The VERB NOUN type tags, that go in the .pos attribute, are from Universal Dependencies, and are mostly the same between languages. The coarse-grained tags, for the .tag attribute, can be anything and are unique to each language as far as I'm aware.

How about this: