is the relational database
When the database was invented computer science became a solve problem.
It is a perfect abstraction, a timeless solution.
It was created 50 years ago and yet still solves the data needs of 99% of business applications today
All modern web applications are 95% database and 5% a small wrapper layer on top to provide authentication and formatting.
Just setting up a database already solves 95% of the functionality of the software.
The database is the most beautiful abstraction in computer science
Yes is amazing, but academia ignore it, your startup culture ignore it and people believe is just boring and old knowledge.
Who is ignoring it? It is used in almost every company and has even outgrown the computer science departments with it being taught in business and data science courses.
>how limited they are
What more would you want a computer to do. Humans are organisms that accept information transform and manipulate it and then act on it. The SQL database is the part of that process that accepts data and organizes it. It also performs some data manipulation if you want. Data is the fundamental building block of informed action. Informed action is how we expand the potential horizon of humanity.
Yeah but usually not as well.
based thread
BASED.
I'm committing this post to my FOSSIl repo right now!
Is the fact that databases are used for 99% of web applications a reflection of how versatile databases are, or a reflection of how limited computers really are? I feel like it's the latter.
Most people seem to have the idea that computers can "do anything" when really the set of problems they solve is extremely limited.
Computers can store data and retrieve data. And perform calculations. And can then optionally send those to another computer. And that's literally it.
In fact computers don't really do anything more than pic related. But the calculator is really fast and you can share notebooks between people instantly.
>calculator notebook
Where can I find one of these for cheap with an actually decent calculator in it?
>Computers can store data and retrieve data. And perform calculations. And can then optionally send those to another computer. And that's literally it.
And that's literally all you need. In future when we will be able to simulate entire universes and live in them it will still be all you need computers to do, just scaled up.
Smart people invented the stacks with sql in them. They just use them because they were told and don't really know.
WordPress is a great example. 98% of WordPress sites would be better as SQLite as well.
these feel like bot posts to me
You're not wrong, but ultimately that's just all a lot of problems and questions in life come down to: organizing, interpreting and spreading information. Computers do those three things very fast, at high volume, and, most importantly, consistently.
>computer
>science
kek
Getting semi-decent at relational databasing has been one of the most important things in my career. I've somehow ended up as being "the guy who knows databases well" while feeling like my knowledge is pretty shallow compared to any semi-competent DBA. I considered trying to pivot from developer to DBA several years ago, but a friend who has been around for a while observed that, when there is downsizing, DBAs are usually towards the front of the line for the chopping block with the rationale that the developers can also manage the database.
>rationale that the developers can also manage the database.
well, can they?
>when there is downsizing, DBAs are usually towards the front of the line for the chopping block >developers can also manage the database.
I've seen this tried, and the result is that what was once a cohesive system with a well-defined schema, is now basically loosely-related JSON blobs with index keys. So it works, it's shit, and it will probably explode in a few years, but sadly the DBA jobs aren't coming back. The fricking webshitters won.
and aws runs what % of databases?
it's funny that faunadb billed itself as the next gen database for the next twitter. not realizing centralization kills all. it's great that compute and data can be right next to each and have a serverless fling but I don't want to pay for every 4kb page read, you dumb fricks will double dip behind the scenes by evicting my big hot stuffs from memory. back to baremetal I go.
>and aws runs what % of databases?
Single digits. Only like 10% of the internet is on the cloud. Databases are even fewer due to the amount of protected data. Pretty much all banks and telecomms are still on self hosted databases. I think there are some exceptions but not a lot.
Good post, OP
ok I feel like this may be a stupid question but why don't we just use file systems with something built on top of it?
They both store files.
Or is it just that it was maybe simpler to just build something from scratch for caching, primary/foreign keys and so on?
Also why is it considered bad practice to store audio and videos and such large binaries in sql databases
databases do https://en.wikipedia.org/wiki/ACID better
Why don't we use databases as filesystems then.
because filesystems aren't meant to be ACID, they're more primitive
>why don't we just use file systems
databases are much faster because of how they organize bits
Filesystems are intended to be a much more primitive and "low level" way to store data. When stripped of all metadata, a file is just a sequence of bytes.
They are not a substitute for databases, in fact, the lattet are usually implemented on top of the former.
Yes it is Op, and even more beautiful are Geospatial database. Love my Postgis till I die.
You are 100% right and I'm tired of fricking morons trying to re-invent the data layer with all the modern garbage.
Yes, it is, and I'm tired of pretending that it's not!
Heidegger lost the battle to Goedel. Set theory is more broken than a Turing like system. SQL is also broken because it contains Turing like syntax which is slower than running the same algorithm in an ORM and doesn't use the set theory that made SQL the "killer app" of mainframes when it was first released.
i wasn't sure whether this was a troll or just a pseud vomiting words, but then i read:
>SQL is slower than an ORM
good bait, almost had me, 8/10
It is slower if you use loops and conditional statements. What I clearly said was Turing like syntax. If you're all you use are SELECT, COUNT, UPDATE, etc then SQL is better. But most databases are not optimized to run Turing like languages as well as Turing languages like Java, C#, C++, etc
>PL/SQL is SQL
now anon is baiting even harder
If you ask DBAs you'll find that 80% aren't able to tell the difference.
what battle did gödel and heidegger have lol?
Heidegger wrongfully thought that set theory could prove mathematics. He was wrong and Goedel was the man who proved him wrong. Lookup godelian sentences
you're thinking of bertrand russel, anon
heidegger was the one who wrote about tools, he didn't say much about maths proper
and the russel/godel thing wasn't a "war," godel wrote well after russel died
No, Bertrand was the guy who said "The set of all sets that contain no sets". Goedel was the guy who formalized it and gave a concise answer to the entscheidungsproblem.
yes, exactly, neither of them are heidegger.
>Bertrand was the guy who said "The set of all sets that contain no sets".
"The set of all sets that don't contain themselves." was more like what he said. The point was he defined a construction that was a contradiction, and so impossible to ascribe a proper mathematical meaning, demonstrating that there were things that lay beyond what conventional math could describe.
Godel took that idea and used it to show that all of mathematics was necessarily incomplete: that there was no finite set of axioms from which all the rest could be derived.
Turing showed that this was equivalent to it being impossible to write a program to prove whether an arbitrary program halts. (He was talking about algorithms, but we'd probably call them programs now.)
The point is, in a period of a few decades at the start of the 20th century, we went from thinking mathematics was finite and knowable, to knowing it is unbounded in scope and that there will always be more to learn. We also learned that programs and proofs are isomorphic and that there are infinitely many interesting programs possible.
I've both studied Heidegger and Gödel a bit and have never heard of anyone mention that there was some kind of battle between Heidegger and Gödel and while Heidegger may have mentioned set theory somewhere he didn't really bother himself much with proving foundational mathematics in the same sense as the frege, russel and that whole analytical gang. Heidegger was a continental philosopher. Wtf are you talking about?
Honestly guys its a google search away
https://online.ucpress.edu/representations/article-abstract/134/1/116/1714/Heideggerian-MathematicsBadiou-s-Being-and-Event
"heideggerian" means "inspired by heidegger" in this context, not "written by heidegger." the source you linked isn't even about heidegger; it's about baidou. you're obviously spouting off a lot of verbose nonsense and ad-hoc googling to justify yourself.
I didn't mean a literal duel between Heidegger and Goedel. Christ what were you picturing? Two Prussian officers.
they worked in completely separate fields anon, stop trying to justify your nonsense
This isn't nonsense. There used to be serious academic thought that mathematics and philosophy were connected through logic. The result of this thinking was set theory. Goedel proved that it isn't possible to have a single set of axioms which prove all of mathematics.
You aren't as well read as you think you are if you aren't aware of this.
>Heidegger lost the battle to Goedel
>Heidegger wrongfully thought that set theory could prove mathematics. He was wrong and Goedel was the man who proved him wrong
>No guys trust me on this, look at this essay that draws an allegory between and interprets Heidegger's continental philosophy as set theory or something. It's written by some random dude's second hand interpretation of Alain Badiou
>no you misunderstand, I didn't mean an actual battle as in an intellectual conflict with communication between the two. I just meant this random research article that you need to be logged in to read
Oh for Christ's sake I'm going to dumb this down into a TED talk for you. You obviously don't understand what Goedel was trying to prove.
I know what gödel proved but you're speaking like an absolute idiot anon
>I know
Now repeat, what was he trying to prove.
I didn't watch the video but a bunch of mathematicians working within the foundations of mathematics were obsessed with finding a complete mathematical system, such as Frege with his set theory. Gödel proved with his incompleteness theorem that wouldn't work. Or whatever.
Now I'm still waiting for this supposed battle between Heidegger and Gödel because I can't find a source of them referencing each other.
and the more I research this and try to find anything about this supposed battle and read up on Gödels philosophical views documented by Wang, Gödel seemed to have quite liked the study of phenomenology. He didn't approach it from a scientific way though because that's in continental territory and a systematic model that's built different from a scientific one.
Speaking about a non-existent conflict, hand-waving while telling me to look up gödel sentences and that the answers to my questions in relation to what you're saying yourself can be found in a closed research article and then sending me a ted talk and wanting me to explain what Gödel proved isn't selling me on your argument but I'm still very much open to hear you out and for you to provide me with something. Thank you.
no it's not.
>a text that uses Heideggerian metaphysics to perform an allegorical exegesis of mathematical set theory and does so as a means of fashioning a supremacist spiritual pedagogy for a philosophical elite in the context of a national intellectual subculture.
it's just some guys writing and second-hand interpretation. It's not a battle between gödel and heidegger.
moron. Godel's theorem just shows us that ALL mathematical proofs cant be derived from the same axiom set. We can have alternative axiom sets for various purposes.
SQL is a dead simple application of set theory. Actually anon mathematicians still use set theory all the time. It's not broken.
You don't know what your talking about with Heidegger. He has nothing to do with this. Godel's work is a response to Principia mathematica which is work by Russell and whitehead. Go frick yourself and stop talking about this shit you haven't read. YouTube essays aren't not knowledge anon.
relational databases are based but most SQL implementations are garbo
SQLite is the only one that doesn't annoy me
Taking a database class seemed to make software click for me.
It really does seem like data is the atom of software.
Based OP seeing the beauty of software's first principal.
That being said, I am curious about graph databases.
>graph databases.
A relation is the most efficient way to store and query a general graph. Read the original relational database paper.
The OOP boomer fears the Spark
Spark is a colossal waste of money and time for any structured data under 1TB.
>data under 1TB
literally just use one big JSON file lmao
1TB is too little for Spark and not significant for the SQL standards.
eh, if you're doing large analytical workloads 1TB is right around where some of the SQL DBs start to have issues and spark starts to shine. For transactional yeah, nowhere near the max.
No, 1TB is just fine for most database engines. PostgreSQL with the TimescaleDb & ZFS Filesystem extensions can handle queries with a dataset of roughly 10TBs of data without an issue.
Not in near real-time it can't aside from 1-2 queries that you've indexed the hell out of it to squeeze blood from the turnip
Actually it can, in fast enough time (reasonably enough) and for large querying/sorting. You don't know what you're talking about.
Fast enough != near-real time and you know it
It can be used for 20 and 30GB datasets too. Spark is more of a hundreds of terrabytes to Petabyte scale.
It's more about data streams nowadays. Process and throw away.
My favorite databases were antique IBM tabulation systems based in Central Europe during the mid 20th century. They were not relational, but they were beautiful.
This is true, but stored procedures were a mistake. Please, keep your processing logic outside of the database.
>smart contracts
the relational model is a solid concept, existing implementations are flawed.
https://en.wikipedia.org/wiki/The_Third_Manifesto
Where can I read more about the history of database development/theory? This thread has given me a new found interest
mostly just look through Wikipedia articles, the database stuff is pretty thorough and cites good books for if you want to dive deeper.
For real. Virtually every case of inheritance hierarchies I've seen in high level code could be much better served by proper use of a database instead. The actual program logic can and should be as "flat" as possible, delegating as much as possible to data.
Data is far more important than code, every developer worth her salt eventually comes to realize this. Build a proper representation of your data and 90% of your problem is already solved.
Incidentally, this is why I hate OOP so much. OOP replaces data with code. What is supposed to be a transparent collection of easily modified data, decoupled from the core logic of your program, becomes an black box of lasagna that only increases costs.
What is a good SQL book for someone who doesn't know anything about databases? Internet tutorials that I can find are low quality, and not very edifying I think.
sql really is best learnt just diving in.
Alan Beaulieu
Learning SQL: Generate, Manipulate, and Retrieve Data
Going through this one right now and learning lots of things. Might be best to speedrun the basics from some other resource like I did, but it's not necessary.
What's the best open source database if I want efficiently updated materialized views?
https://www.emacswiki.org/emacs/SqlMode
...is a really comfy way to write SQL and send it to a running SQL repl.
Emacs is truly a gift from G-d, I feel sorry for those who can't appreciate it.
what if instead of perfectly designed, well thought out relational databases with cohearant schemas OP, what if we just took random data, shoved it into a javascript object and just stored billions of them?
Can I do that with NodeJS?
sounds like a triple (a set of three entities that codifies a statement about semantic data in the form of subject–predicate–object)
[200, movie/title, "The Terminator"]
[200, movie/year, 1987]
https://www.instantdb.dev/essays/datalogjs
I work with this shit for my job and I still have no idea how it works.
https://urbit.org/docs/arvo/overview#acid-database