A Relational Model of Data for Large Shared Data Banks

Here is a line of Python you could write in any Django app today. It asks for the part number, the part name and the quantity committed, for every part used in the project named alpha:

Commitment.objects.filter(project__name="alpha").values_list("part__number", "part__name", "quantity")

Django, an ORM (a library that turns database rows into objects in your code), turns it into SQL with two INNER JOINs it wrote for you: steps that match rows in one table with rows in another. The database hands back the rows (in our example data, a bolt, a gear and a spring). Look at what you never said: which file holds the parts, what order the rows are kept in, which index to use, or how to get from a project to its parts. Why are you allowed not to know any of that, and what did it cost before you were? In 1970 this exact question, almost word for word, is the example in an eleven-page paper by Edgar F. Codd. Back then, answering it meant a program that walked from record to record, and that program broke whenever the storage changed shape.

The Problem in 1970

Codd was a mathematician at the IBM Research Laboratory in San Jose, California. His paper appeared in Communications of the ACM in June 1970 (volume 13, number 6, pages 377 to 387; received September 1969, revised February 1970).

The systems he describes give users "tree-structured files or slightly more general network models of the data." IBM's own product, IMS, is one he cites. In a tree, a project record sits on top and the records for its parts hang under it. To answer a question, a program starts at one record and follows the links down, record by record. So the program has to know exactly how the data is laid out, and Codd notes that such programs "fail when a change in structure becomes necessary."

The paper opens by saying that this has to end:

Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation). …

… Activities of users at terminals and most application programs should remain unaffected when the internal representation of data is changed and even when some aspects of the external representation are changed. …

(p. 377)

He calls the goal data independence: "the independence of application programs and terminal activities from growth in data types and changes in data representation." In the acknowledgment he credits C. T. Davies of IBM Poughkeepsie with convincing him of the need for it.