We do teach these things, they are just not core CS topics, but rather in other areas, relegated to electives like a software engineering course. At CMU we have entire Master's program for software engineering and an entire PhD program (in my department). We teach exactly the kinds of things the blog post is about, and more. Software Engineering is a whole field, a whole discipline.
I get that this is a blog post and needfully short, but yes, there are courses that teach these skills. There's a big disconnect between CS and SE in general, but it's not as bad as "no one teaches how to build quality software". We do work on this.
We were taught these things at the Bachelor's program in CS I went to in Sweden. At my first job I then slipped on a banana peel into a de facto tech lead position within a year, and I don't think it was due to my inherent greatness but rather that I was taught software engineering and the other colleagues at my level had not.
Ironically, the software engineering courses were the only ones I disliked while a student. An entire course in design patterns where strict adherence to UML was enforced felt a bit archaic. We had a course in software QA which mostly consisted of learning TDD and the standard tooling in the Java ecosystem, with some cursory walkthroughs of other types of QA like static analysis, fuzzing and performance testing. At the time it felt so boring, I liked to actually build stuff! A couple of years later I joined a team with very competent, CS-educated developers tasked to set up a workflow and all the tooling for testing software with high security requirements. They were dumbfounded when I knew what all the stuff they were tasked to do was!
There's a massive gap between taught at CMU and taught at most universities. And even if it is taught, it's usually outdated or focused on very literal stuff like how to write web applications. I'd have killed for a class that actually focuses on implementation, on teamwork, on building complicated systems.
Almost every CS course I took went the other way and had strict cheating policies that essentially made any group work verboten. There was 1 group project in 1 course I took in 4 years.
My spouse on the other hand took an explicitly IT program and they had group projects, engaging with real world users, building real solutions, etc.
That's crazy! My uni had senior design projects, and labs. Shoot, my community college had those, too. Most of our classes had lab projects, and sometimes the lab projects were group projects. We were encouraged to help each other out. I can't imagine a class where collaboration was totally against the rules.
And I mean, that went with everything. In my EE classes we did lots of collaboration. We had study groups, homework groups, etc. It was a lot of fun. I'm sad to hear there are places where they straight up make that against the rules.
My uni also had engineering ethics classes - in my major it was mandatory to take 2 of them. I think it makes sense and should be more common for software engineers. A lot of software is used to operate planes, cars, medical equipment, and nowadays also help make decisions that can have life-threatening consequences.
> strict cheating policies that essentially made any group work verboten
If I had to guess, some polytechnic school or another?
With some classes even forbidding discussing work with other students, where each assignment required a signed (digitally or otherwise) affidavit listing everyone you consulted, acknowledging that if you actually listed anyone, you were admitting to violating the academic honesty policies and if you didn't list anyone yet had spoken with others you were of course also violating the academic honesty policies.
Where only consulting the professors or TAs was allowed, the TAs were never around, and the professors refused to help because if they gave you any hints, it would apparently give away the answer, which would be unfair to the other students.
I had students that copied one-another's work; in fact I had few students that didn't copy. It made it impossible to mark their projects correctly, so I asked my more-experienced colleagues.
The best advice I got was to explain the difference between collaboration (strongly encouraged) and plagiarism (re-take the course if you're lucky). Forbidding collaboration is a disastrous policy, so an instructor shouldn't have a problem with giving the same mark to each member of a group of collaborators. You just have to test that each individual can explain what they've submitted.
My school auto-graded everything (file drop your code to a server by midnight & get a score back). I don't recall a single instance of TA/professor written or verbal feedback on code.
Yuh. I guess that's "modern times". I taught in the late eighties, and auto-grading wasn't a thing.
FWIW, I was a "temporary visiting lecturer", i.e. contract fill-in help. I had to write the syllabus (literally), the course plan, and the hand-outs. I also had to write all the tests and the exams; and I had to mark the exams, and then sit on the exam board through the summer holidays. The pretty girls would flutter their eyelashes at me, and ask for leniency. I suspect they took their cue from my louche colleague, who was evidently happy to take sexual advantage of his students in exchange for better marks.
[Edit] I liked this colleague; but I would not have introduced him to my wife or my daughter.
This was in the 1980's. We're talking 5 1/4" floppies, no internet, 64MB RAM. I had to review my students' work as dead-tree submissions or completed circuits or whatever.
(I certainly wasn't going to take digital submissions from them; that would mean floppy disks, and I regarded any disk that had been touched by any student as if it were infected with plague, because it almost certainly was. All the school systems were always infected).
I taught a course "in a previous life" and while I wasn't anything close to as strict as you say here, I can tell you the flip side: students would copy, modify superficially (some, less superficially) and then claim "it's my work, we just talked, that's why the similarities!" (with some even having the nerve to say, "it's not copied, look, plagiarism detection software says similarity is less than x%!)
Perhaps I was wrong but I really wanted the students who took the course to put in the work themselves. Just choose a different course/course set, you _knew_ this one was going to be hard!
So yeah, the guideline was that we'll be eventual judges of what was "too similar" and if you're concerned, just don't discuss implementation details with anyone. I realize it prevents honest colaboration, and that's bad too... but sometimes it's a "damned if you do, damned if you don't" kind of situation.
What we did when correcting the homework is compare the signature of the assembly output (not manually of course). You can move functions around, rename them, change the names of variables, .... but the signature of the instructions remains the same.
We caught 2 guys, of course we didn't know who copied from whom, but we quickly found out by challenging each of them with a fizz-buzz kind of question.
Also, for whatever my purely anecdotal experience is worth, I'd say the general quality of a professor's teaching was negatively correlated with how hard-ass they were about anti-collaboration policies. That also held true for a couple classes I'd dropped and retaken with a different prof.
One might think I'm just rating easier professors higher, but no, there were definitely enough sub-par teachers with lax policies and forgiving grading who failed to impart what their course was supposed to cover. There were also tough professors I learned a lot from. It's just that I can't recall a single decent instructor among those obsessed with anti-collaboration policies.
I've taken course work at 6+ different universities, and in my experience different groups of international students have very different perspectives on what is cheating vs. collaboration. I think it's likely attributed to the western ideal of the individual vs. the collective.
My university did this - each semester's class inherited the code base the last semester had worked on.
As a learning tool it was a disaster. The code had never been touched by a skilled engineer, just years of undergrads pulling all-nighters.
That meant it didn't teach what good maintenance looked like. Students just piled their hacks on top of someone else's hacks so they could hit the deadline.
It wasn't a good way to learn to write quality software (even if reality sometimes looks this way).
It was also the only software engineering course in our CS curriculum.
Not each week, but at my university, they had enough focus on the science part that most semester long solo projects were semester long and solo only in name. Students were assigned some component or incremental stage of some larger project spanning years and multiple PhDs, not some assignment for assignment's sake. That certainly did not make them perfect learning environments because students were left to their own in how they solve cooperation (postgrads running those macro projects might be considered PO, but not technical leads, and just as clueless as the rest) but the "pass on to the next each week" approach would have exactly the same gaps.
Me too. All throughout my education, group projects and speeches were both these scary, uncommon things that you could afford to fuck up, and I skated by on individual talent.
Now I'm an adult, several years into my career, wishing that the schools had done a bigger quantity of low-stakes group projects instead. It's a muscle that was never exercised
that would be a cool semester long project assignment: everybody has to plan/architect their own software project, and then work on implementing them, but you don't work on your own project, that you just manage.
This just isn't realistic in an general CS undergrad though. Students don't have the time or foundations to build a realistic complicated system, and the institutions don't have skills or currency to teach them how. You complain about content being out of date, but the foundations of quality software are not by and large technical nor go out of style. The implementations sure do.
What you're asking for is where vocational (2 year) programs really shine, and one of the few places where I've seen bootcamp graduates have an advantage. Unfortunately both then lack some really valuable underpinnings, example: relational algebra. It seems that once again there is no silver bullet for replacing time and experience.
In software engineering schools in France, both in my personal experience and other Bac+5 people I’ve hired, the entire Bac+4 year was dedicated to group projects. When you have 8 group projects in parallel at any time, it does teach team work, management, quality control, all the good stuff.
After the first failed project, everyone agrees that they need a boss without assigned duties. Then in 5th year, we’d have one or two class-wide projects (20-30 people).
This, along with joining event-organizing charities on the campus, where you’d work 3-5hrs a week to build a hundred-thousand-to-a-million dollars event (NGOs on campus, or concerts, or a sports meeting, to each their preferences, but it was the culture). And it wasn’t always a good time together, it’s not flowers all the way down.
I’m only surprised some school might consider engineering without group effort. On the other hand, I deeply deeply regret spending this much time in charities and not enough dating, as I have never been recognized for charity service, and family was important to me.
Yeah. A fair bit of this is just “people working in teams” stuff, that people that buy into ‘developer exceptionalism’ will tell you is sacred software developer knowledge. It isn’t.
Software engineering isn’t just about teamwork, and not all software development-related teamwork skill is generalisable to other industries, but it’s far from uncommon for there to be some trendy blog post laying out the sorts of things that, yes, an MBA program will teach someone. Which is fine, if not for the fact that these same people will scoff at “clueless MBAs”.
Do they funnel soon to be grads into stressful zoom calls where product managers handwave an entire legacy stack still somehow running on coldfusion and want a rebrand with 'AI' starting Jan 1??
No, but our professor assigned teams at random, gave us a buggy spec, and then changed the spec with one week to go during finals week. (This last part appears to have planned; they did it every year.)
This was a surprisingly effective course, if sadistic.
I had something like this too - it was required for my CS degree. Our class split up into teams of 5. But the whole class of 30 was working on a single project. It was a semester-long project and each team also had to integrate with eachother to build the final solution.
Here's the thing, though: Of a CS graduating class, 90% of them will work as software engineers, not as computer scientists. (All numbers made up, but I think they're about right.)
We don't need these things to be electives. We don't need them to be a master's program. We need an undergraduate software engineering program, and we need 90% of the people in CS to switch to that program instead.
I agree with you! It's hard to change curricula because there are so many competing interests. CS is an evolving field and things like machine learning have burst onto the stage, clamoring for attention. There is also an age-old debate about whether CS departments are trade schools, math departments, or science. Personally I think software engineering skills are paramount for 90% of graduates. How do we fit this into a full curriculum? What gets the axe? Unclear.
> There is also an age-old debate about whether CS departments are trade schools, math departments, or science. Personally I think software engineering skills are paramount for 90% of graduates.
The question as well is: are Chemical Engineering, Mechanical Engineering, Materials Engineering trade schools?
I think it's a key call out as CS touches on so many things.
There are arguments for it being math, science, engineering, a trade school or a combination of the above.
And then if you separate them out completely you end up with people doing CS being out of touch with what happens in the real world and vice versa.
I think in the end you probably need to have a single overall degree with a specialization in (Programming as Engineering, Programming as Math and Programming as Computer Science,) with lots of overlap in the core.
And then you still can have a both a bootcamp style trade school.
Now all of that said, that still doesn't solve the CS equivalent of "Math for business majors". Or the equivalent of "Programming for Scientists", or the like which is already a really important case to offer. Where you major in Bio/Chem/Other but being able to apply programming to your day job is important.
Although that probably sits closer to the Applied Software category that you might find in business school like using spreadsheets, basic command lines, intro to databases or python.
But to your point, how rarely software engineering is being taught is a huge problem. Even if only 30% of degree holders took classes in it, it would be huge in helping spread best practices.
I don't really think that software engineering is a trade per se. I think it is a much more creative activity that requires a lot more baseline knowledge. I think an automotive engineer is to a mechanic as a software engineer is to an IT administrator. There is still a fair amount of creativity and knowledge required for being a mechanic or IT admin, but I don't think it's nearly the same amount.
Software engineering is interesting, though, because it does not require as much formal education as many other engineering fields to get a job. I think this is in part because it is very economically valuable (in the US at least) and because the only tool you need is a computer with an Internet connection.
With all of that said, I think SWE is probably closer to a trade than other engineering disciplines, but not by all that much.
Exactly what I see. I know a few cases of CTOs influencing the curriculum of CS courses, pushing it to be more “market ready”. Which was effectively turning it into a 4 year bootcamp.
Which is a big shame, because it’s very common to see developers who would have benefited from a proper CS curriculum, and have to learn it later in life.
For years a friend tried to get his department (where he worked as a lecturer) to add a software engineering course where the basic syllabus was to (1) receive a TOI from the last semester and the code base, (2) implement some new features in the code base, (3) deploy and operate the code for awhile, and (4) produce a TOI for the next semester.
The code base was for a simple service that basically provided virus/malware scanning and included the malware scanner and signatures (this ensured there would never be an end to the work - there's always more signatures to add, more features, etc.)
I thought this was a fantastic idea and its a pity he never convinced them. That was more than fifteen years ago, and in his plan it would have just run forever.
In my university, (US, state school,) we had a software engineering course exactly like this. It was great in theory, but in practice, the experience was rushed, the codebase was poor quality, (layers upon layers of nothing features with varying code quality,) and the background knowledge was completely ignored. The application we had to work on was a Tomcat Java Web application with an Angular frontend, when neither of those technologies were taught in any other classes (including electives.)
This approach to education can work, but I think simulating/mocking portions of this would have been more helpful (it could've been a teacher/TA managed codebase we started with rather than the monstrosity passed between generations of students who were inexperienced.)
You needed to partner with the business school to get a future MBA to convince the faculty (executives) the biggest return (profitability) was a total re-write!
I had a single phone interview with someone at Northwestern (a long time ago) where they were looking for someone to build a pen of developers to "partner" with MBA students to turn ideas into apps. I laughed so hard my sides hurt.
I think like your example where things go wrong is the most realistic exposure to programming you can give someone.
Learning why things are bad, and why it's bad to experience them offers a new level of appreciation or better ways to argue why certain things should be done.
The thing is for academics quality software sometimes isn't actually quality software.
My experience has been that people who's first jobs were in companies with quality software or who's job included reading through other people's quality software learn to write good software, the other ones learn whatever they saw in the environments they worked in.
That sounds like an excellent way to do practice that directly mirrors real-world SWE, while still cutting it down to an appropriate size for a pedagogical environment. What a good idea.
I went to a STEM school and exactly 0 of the professors had been in industry ever or at least in the last 30 years. The only guy with some experience was an underpaid lecturer. He was also the only good lecturer.
A lot of professors just want to do research and mentor students onto the PHD track to self replicate. My mandated faculty advisor was basically like "go to the career center" when I asked about, you know, getting a job of some sort with my degree.
So yes, it is a real problem. CMU may stand out by actually having courses in the space, but it is not the norm by any means.
If my friends hadn’t had such vividly bad experiences with the compiler class, I might not have taken the distributed computing class that was one of the other options to fulfill that category.
It’s not the most defining class of my undergrad years, but it was pretty damned close.
The fact that most people designing systems don’t know this material inspires a mix of anger and existential dread.
After seeing the same mistakes made over and over again I can't help but agree. This is how one builds enterprise software now, and it is poorly understood by most developers, although that is starting to change. If I were designing a college curriculum, required courses would include all of the normal CS stuff but also software engineering, distributed computing, design patterns, web application fundamentals, compiler design, databases, DevOps/Cloud, testing fundamentals, UX design, security basics, data science, IT project/process management, essentials of SOA, product mgt and requirements. Of course, it's been so long since I went to college, none of these things existed back in the day, so perhaps modern curriculum has all of these now!
I took one of these kinds of classes in my masters program this year. They were totally obsessed with UML. It would be nice if these classes could move beyond dogma that is decades old.
What would be better? Change tools every 3-5 years like the industry does, so by the time any given instructor actually has a grasp on a particular tool or paradigm, its already obsolete (or at least fallen out of fashion) too?
I'm no fan of UML, but the exercise is to teach students how to plan, how to express that plan, and how to reason about other people's plans. The students will certainly draw a lot of flow diagrams in their careers, and will almost certainly deal with fussy micromanagers who demand their flow diagrams adhere to some arbitrary schema that has only limited impact on the actual quality of their work or documentation.
I haven't seen a UML diagram once in 7 years of working. The approach presented in the book "a philosophy of Software Design" is much better than the outdated bullshit from the 90s.
I never got the hate against UML. To me, UML is just a language to communicate about several aspects of a technical design, and to visualize a technical design to make it easier to reason about it.
I did not read the book "a philosophy of software design", but I just scanned the table of contents, and it is not clear to me how "a philosophy of software design" contradicts with using UML.
Are you telling me that in those 7 years of working, you never once used a class diagram? Or a database diagram? Or an activity diagram, deployment diagram, sequence diagram, state diagram?
I find that hard to believe... how do you explain your design to other people in the team? Or do you mean that you do make that kind of diagrams, but just use your own conventions instead of following the UML standard?
Personally, I often use UML diagrams when I am working on a technical design and I use those diagrams to discuss the design with the team. On almost every project I create a class diagram for all entities in the system. I rarely make a database diagram because that often is a direct translation of the class diagram. For some of the more complex flows in the system, I create an activity diagram. For stuff that goes through state transitions, I create state diagrams. In my case, this really helps me to reason about the design and improve the design. And all those diagrams are also very useful to explain the whole software system to people who are new on the project.
That does not mean that my diagrams are always strictly UML-compliant or that I make a diagram for every little piece in the software system. The goal is not to have UML-compliant diagrams, but to efficiently communicate the technical design to other people, and UML is nice for this because it is a standard.
UML is a tool to do something (namely, formal and detailed specification of systems) that in many places nowadays isn't really done. I think it is totally plausible that over 7 years of professional work OP has never been in a situation where one person has made a detailed design and wants to present it in a formal manner to other people in the team using diagrams (as opposed to answering specific questions of "what does this particular thing in the code do"). If they discuss the process, they tell about the process without using an activity diagram. If someone wants to view a database diagram, they use some tool that autogenerates one from the data, and discards it after viewing instead of attempting to maintain it as a formal documentation.
I agree that all those diagrams are also very useful to explain the whole software system to people who are new on the project, however, that doesn't imply that having this ability is common, many (IMHO most, but I don't have the data) companies intentionally don't put in the time and effort to maintain such up to date diagrams for their systems.
> Change tools every 3-5 years like the industry does, so by the time any given instructor actually has a grasp on a particular tool or paradigm, its already obsolete (or at least fallen out of fashion) too?
I mean, yeah. Seeing that wave happen over the course of their college career would probably be better prep for a career than most CS classes.
Great! So, do you bring your whiteboard to lecture to turn in, or take a picture of it, or just schedule time with your professor to whiteboard in front of them?
All I'm arguing for here is that UML serves the same purpose as those online homework apps. Correctly formatting your calculus homework to be accepted by that interface is as unrelated to calculus as UML mastery is to effective software design, but it resolves some of the same logistical challenges, while introducing others.
A whiteboard is just a medium to draw.
Uml is a standard that says how to express certain ideas as symbols that can be drawn.
It's not clear to me what your argument is.
Is it using whiteboards to draw uml instead of special uml software?
If so, be prepared to take much longer to draw the diagram.
Or do you mean uml is deficient compared to free drawing of symbols on a whiteboard without a standard?
If so, be prepared that nobody will completely understand your diagram without explanation
CMU constantly reevaluates its MSE program with input from many different angles. I've participated here and I think we're trying hard to balance important foundational knowledge with practical skills of the day. I don't think we over-emphasize UML or any one particular silver bullet in our program.
To a first approximation, software developers don't have masters degrees. If you are thinking about changing how an industry does its work, focusing on graduate courses seems counterproductive.
I disagree. I have a Master's in Software Engineering and the way to change things is for those with the formal education to try and spread good practices as much as possible in the workplace. Sometimes the main benefit is just knowing that good practices exist so you can seek them out.
The biggest impact I've had at the places I've worked have been about procedures and methodology, not how to use UML or draw a dataflow diagram.
- Have a process around software releases. Doesn't matter what as much as it has to be repeatable.
- Review your designs, review your code, don't do things in isolation.
- Have a known location for documents and project information.
- Be consistent, don't do every project completely differently.
- Get data before you try to fix the problem.
- Learn from your mistakes and adjust your processes to that learning.
- And many more things that sound like common sense (and they are) but you'd be amazed at how even in 2023 many companies are developing software in complete chaos, with no discernible process.
What I'm saying is that if your goal is to introduce more engineering rigor and your plan is for for the tiny percentage of graduate school graduates to percolate these ideas through the industry, it's probably a bad plan and likely to fail.
This was a thread about why software developers don't do engineering like other disciplines. One partial answer is that those other disciplines take it much more seriously at the undergraduate level, at least on average.
Probably the more compelling answer is that the industry doesn't' really want them to for the most part.
While I'm not suggesting that UML is necessarily the solution, I hope it's not, but the observation that so few developers touch anything that just looks like UML is a good indication that a lot of software is in fact NOT designed, it's just sort of hobbled together from a few sketches on a whiteboard.
I hope that people say they hate UML and then just make UML (class, database, activity, ..) diagrams according to their own conventions, but I am afraid you are right and that a lot of software is just "hobbled together"...
But debugging is about "trying out random things". You can call it a Monte-Carlo tree search is you want to sound smart.
And I don't feel is not something that is worth teaching in universities, because it is 90% experience and for me, the point of universities is not to replace experience, just give enough to students so that they are not completely clueless for their first job, the rest will come naturally.
What universities can teach you are the tools that you can use for debugging: debuggers, logging, static and dynamic analyzers, etc..., different classes of bugs: memory errors, the heap, the stack, injection, race conditions, etc..., and testing: branch and line coverage, mocking, fuzzing, etc... How to apply the techniques is down to experience.
In fact, that's what I find most disappointing is not juniors programmers struggling with debugging, this is normal, you need experience to debug efficiently and juniors don't have enough yet. The problem is when seniors are missing entire classes of tools and techniques, as in, they don't even know they exist.
The "Monte-Carlo tree search" space is usually far too large for this to work well!
It is true that initially you may not know where the bug is but you have to collect more evidence if possible, see if you can always cause the bug to manifest itself by some tests, explore it further, the goal being to form a hypothesis as to what the cause may be. Then you test the hypothesis, and if the test fails then you form another hypothesis. If the test succeeds, you refine the hypothesis until you find what is going wrong.
Such hypotheses are not formed randomly. You learn more about what may be the problem by varying external conditions or reading the code or single stepping or setting breakpoints and examining program state, by adding printfs etc. You can also use any help the compiler gives you, or use techniques like binary search through commits to narrow down the amount of code you have to explore. The goal is to form a mental model of the program fragment around where the code might be so that you can reason about how things are going wrong.
Another thing to note is you make the smallest possible change to test a hypothesis, at least if the bug is timing or a concurrency related. Some changes may change timing sufficiently that the bug hides. If the symptom disappears, it doesn't mean you solved the problem -- you must understand why and understand if the symptom disappeared or the bug get fixed. In one case as I fixed secondary bugs, the system stayed up longer and longer. But these are like accessories to the real murderer. You have to stay on the trail until you nail the real killer!
Another way of looking at this: a crime has been committed and since you don't know who the culprit is or where you may find evidence, you disturb the crime scene as little as possible, and restore things in case you have to move something.
But this is not usually what happens. People change things around without clear thinking -- change some code just because they don't like it or they think it can be improved or simplified -- and the symptom disappears and they declare success. Or they form a hypothesis, assume it is right and proceed to make the "fix" and if that doesn't work, they make another similar leap of logic. Or they fix a secondary issue, not the root cause so that the same bug will manifest again in a different place.
I suspect that GP was talking about some notetaking tactics to systematically narrow things down while throwing educated guesses against the wall. Because so much of debugging is running in circles and trying the same thing again. No amount of notetaking can completely remove that, because mistakes in the observation are just as much an error candidate as the code you observe, I'm convinced that some "almost formalization" routine could help a lot.
Good points on the tool side. While "debugger driven development" is rightfully considered an anti-pattern, the tool-shaming that sometimes emerges from that consideration is a huge mistake.
I worked with programmers around my junior year and some of them were in classes I was in. I thought they were all playing one-upsmanship when I heard how little time they were spending on homework. 90 minutes, sometimes an hour.
I was a lot faster than my roommate, and after I turned in my homework I’d help him debug (not solve) his. Then I was helping other people. They really did not get debugging. Definitely felt like a missing class. But it helped me out with mentoring later on. When giving people the answer can get you expelled, you have to get pretty good at asking leading questions.
Then I got a real job, and within a semester I was down below 2 hours. We just needed more practice, and lots of it.
This is why internships and real world experience is so important. A course is 3 in class hours a week over 12-14 weeks typically. After homework and assignments it is ultimately maybe 40-80 hours of content.
Which means you learn more in one month of being on a normal, 40 hour workweek job than you have in an entire semester of one course.
Not all hours are created equal. This is on the verge of saying “I took 1,000 breaths on my run, so if I do that again, it’s like going for a run.” Just because you’re measuring something, it doesn’t mean that you’re measuring the right thing. You’re just cargo-culting the “formal education is useless” meme.
Were you the sort of person who responsibly worked a little bit on the assignments over the course of the week/two weeks, or did you carve out an evening to try to get the whole thing done in one or two sittings?
My group did the latter. I think based on what we know now about interruptions, we were likely getting more done per minute than the responsible kids.
Including reading, we might have been doing 15 hours a week sustained, across 2-3 core classes.
But these were the sort of people who got their homework done so they could go back to the ACM office to work on their computer game, or work out how to squeeze a program we all wanted to use into our meager disk space quota.
Anything more than a B was chasing academia over practical knowledge. B- to C+ was optimal.
I believe that software-related college degrees are mainly there to get the horrible first few tens of thousands of lines of code out of people before they go into industry.
What do you mean by people trying random things? I think that approach (if I understand the term correctly) is more or less what debugging is as a form of scientific investigation.
If you observe a car mechanic trying to find the problem with a car, he would go like: "is this pin faulty? No. Is the combustion engine faulty? No. Are the pedals faulty? Yes." where the mechanic starts with some assumptions and disproves them by testing each of those assumptions until (hopefully) the mechanic finds the cause of the fault and is able to fix it. Similar types of investigations are important to how natural science is done.
So it would be helpful if you can clarify your intended meaning a bit more. Maybe I or someone else would learn from it.
Trying random things seems to be how a large number of professional software engineers do their jobs. Stack Overflow and now CodeGPT seem to contribute to this.
I'm not sure if software engineering classes in particular do, but at my university, they teach C++ in the second required course, and they teach you about using GDB and Valgrind on Linux there. They don't explicitly teach you about systematically debugging, though, beyond knowing how to use those two programs.
Rose Hulman is an undergraduate teaching school that also has a distinction between a Software Engineering Degree and a Computer Science Degree. The Software Engineering degree takes a few less math courses and instead takes classes on QA/Testing, Project Management, and Formal Methods
I chose software engineering. 3 years into the program the head of the department made a speech at an event to the effect of "Software hasn't changed in the last 10 years". It instantly devalued the entire program for me.
I have news for you... He's not wrong. The porcelain is different, but the same methodologies and processes are in place. The biggest change recently is distributed (mostly ignored) version control, that's 20 years old, and continuous integration/development (probably also around 20 years old, but only catching on in the last 10-15 years).
Computer science has changed more, there are lots of developments in the last 5-10 years.
The biggest change I’ve seen in 20 years is that things like DVCS and actual good tool chains for languages people actually use are available and in use.
If you switch to a different framework that does the same things slightly differently and makes something more convenient, and do that three times over the years, that's still perfectly consistent with "Software hasn't changed in the last 10 years" - it's simply not a meaningful change, nor would be switching to a different programming language.
I know where you come from and I know where the people who are responding to you come from too.
Software has changed in the last 10 years, but a lot of it has changed superficially. A green software engineer most likely won't be able to tell the difference between a superficial change and a fundamental change.
It has a lot to do with the topic of this thread. "Quality Software" It's a loaded term. There's no formal definition, everyone has their own opinion on it and even then these people with "opinions" can't even directly pinpoint what it is. So the whole industry just builds abstraction after abstraction without knowing whether the current abstraction is actually close to "quality" then the previous abstraction. It all starts out with someone feeling annoyed, then they decide to make a new thingie or library and then they find out that this new thing has new annoyances and the whole thing moves in a great flat circle.
That's the story of the entire industry just endless horizontal progress without ever knowing if we're getting better. A lot of the times we've gotten worse.
That being said there have been fundamental changes. Machine learning. This change is fundamental. But most people aren't referring to that here.
> "Software hasn't changed in the last 10 years". It instantly devalued the entire program for me.
As opposed to maths, physics, philosophy, civil engineering, classical studies which have gone through complete revolutions in their topics, problems and study methods in the last 10 years?
It seems to vary a lot from school to school. At my university (2010-2014) we were writing unit tests from the first CS class. I can't say we got much instruction on how to structure more complex applications in order to make them testable, however. Things like dependency injection and non-trivial mocking have to be learned on the job, which is a real shame. Even within the industry skills and approaches to designing code for tests feel very heterogeneous.
Not the person you replied to, but I am aware that a functional programming course by CMU has had lectures (not exercises or content) freely available as YouTube recordings. https://brandonspark.github.io/150/
So I have not done research any software engineering field and have not read all that much either. One example that comes to mind from one of my courses that I took in software engineering is research around mutation-based testing. That form of testing is where you generate random variants of your test by doing things like deleting a statement, adding a statement, changing a less than sign to a greater than sign, etc. Then you check to see that at least one of your tests fails for each variant. If it does not, you either add a test or mark that variant as being effectively the same program. I forget what the term is for it. At any rate, I think there is still research being done on this topic, for example how to effectively generate programs that do not generate as many functionally identical programs. Software testing in general is a big part of software engineering, and I think there is still a fair amount of research that could be done about it.
In my opinion, the intersection of cognitive psychology and software engineering is also ripe for a lot of research. I feel like we as software engineers have a lot of intuitions about how to be productive, but I would love to see various things that could indicate productivity measured in a controlled lab setting.
No, mutation-based testing is different. Fuzzing varies the input to the program. Mutation testing varies the program itself as a means of testing the quality of the tests.
Testing, fuzzing, and verification are all pretty hot SE topics now, at least in my department at CMU. All of those have some element of program analysis, both static and dynamic, so there's potentially deep PL stuff, which is core CS. There's aspects to SE that are not just that, of course, which are studying social processes, management styles and practices, software architectures, design patterns, API design, etc.
i feel like the Toyota Production System could be applied, I know they used some of it at Amazon but in general I don't hear much about it in 'software spaces'. It's been studied a huge amount in quality theory but... seems like theres not a lot of crossover between manufacturing and software.
like the idea of a Poke Yoke could cover all manner of automated analysis software tools. from the -Wall in gcc to, Rust borrow checker, basically just Poke Yoke.
The reason I find it easier to work with people who have a degree in Computer Science is that I don't have to convince them of the need for good algorithms and not to try to implement parsers or cryptography by hand.
When it comes to software engineering I feel there is no qualification where you can feel that the gross naivety about quality and working in teams (and with other teams) has been similarly ironed out.
Instead you get people hardened in sin from their repeated experience of writing one horrible bit of untested code quickly and then leaving before the shit truly hit the fan :-)
One's management is very impressed with those kind of people and very contemptuous of those left behind who have to painstakingly clean up the mess.
Hum, interesting perspective. I did 95% of a masters in CS before leaving to do a startup and while I can see the value of parser generators, there are a LOT of times when it is appropriate, useful, and more performant to write your own simple parser. It's often in my opinion the right thing to to first for simple cases. Clearly you should test it and consider functional requirements, but a stringy protocol with clear delimiters is usually dead simple to parse and depending on your use case you can focus on a subset. If you're writing a language... my advice might be different.
I've never had it once in my career where using a parser generator wasn't better. Given that it's an in-language parser generator and not some meta-language monstrum like ANTLR.
Maybe when writing your own programming language, own complicated data format or low communication protocoll while requiring extreme performance. But that seems to be super rare, at least in my area of profession.
I’ve had a very different experience. I can think of three occasions where I was able compare a hand written parser with something built using a parser generator or similar tool. In two of those cases, the hand written code was far easier to read and modify. This kind of code can also be easier to test.
Parser generators aren’t a free lunch and they vary considerably in quality.
Maybe we talk about two different things. I'm talking about libraries that help you write a parser. Those are not tools, there is no extra build-process involved or anything like that.
My impression is the gist is: When you think like an engineer, your focus is on problem solving, and using the appropriate tool(s) to do that. On the other hand, typical developers instinct is to code, code, code; at least based on my experience.
But I also wasn't focused on parser tools. My observation was more universal. That is, engineers look before they leap. Developers leap first and ask questions later. Engineers are intentional. Developers much less so, and far more reactive.
Yeah.. I didn't get rude. Sometimes coders just have the NotBuiltHere attitude. I think it's something you grow out of.
We can build something, or pull something off the shelf. If it takes x time to build it, and then x time to debug and test and x^(1/0) to maintain it; far better to just add a gem. Even if it's not the absolute best, at least it's easy to understand and if it becomes a problem fix the edges.
I've worked with people who thought parsers were straighforward and trying to fix the bugs in their code was fraught with impossibility - there can sometimes be millions of ways for parsers to accept invalid input or not accept valid input.
In one case I gave up on fixing the code where every change introduced new possible bugs and used a parser generator. We never had another bug in that part of the code but my wholesale change caused intense friction.
I feel that a course in parsers would have helped that person to understand this wasn't an appropriate situation.
In fact I think it's a good idea to have BNF "that works" before you hand code anything just to confirm that you understand your own language design.
Cryptography yes, but are you sure about parsers? As far as I can tell, there's some kind of U-curve there. Beginners code them by hand, intermediate-level programmers and intermediate-scope projects use parser generators, and people maintaining the most sophisticated parsers prefer to code them by hand too. For example, GCC used to have a bison parser, but they switched to a hand-coded recursive descent one because that let them produce more helpful error messages. Clang uses recursive descent too.
To be fair though, jetbrains use case is fairly unique, as they basically want to implement parsing for as many languages as possible, all while doing it in a verry structured and consistent way, with having many other parts of their infrastructure being dependent on that parsing API.
I think it's fair to say that those requirements are outside of the norm
I think that's a fine observation, but I'll also add that since their cases are almost always consumed in an editor context, they need them to be performant as well as have strong support for error recovery, since (in my mental model) the editor spends 90% of its time in a bad state. If I understand tree-sitter correctly, those are some of its goals, too, for the same reason
Pushback on parsers. It's very difficult to provide useful diagnostic error messages with yacc/bison. So most languages end up with a hand-written recursive descent parser.
The only exception I personally know of is jq (uses bison). So it's difficult to produce helpful syntax error messages in the implementation of jq.
> The reason I find it easier to work with people who have a degree in Computer Science is that I don't have to convince them of the need for good algorithms and not to try to implement parsers or cryptography by hand.
Cryptography and parsers simply do not belong in the same sentence. There is never a time when it is a appropriate to write your own cryptography. OTOH, most large compiler and interpreter projects have handwritten parsers, and many of them have handwritten lexers too.
Writing a parser can be simple enough to fit into a take-home assignment, and hand-written parser code ends up looking pretty similar to an LL grammar anyway. Parsing is also the easiest part of writing compiler or language tooling, so if a hand-written parser is too high a bar for the team then the entire project might questionable.
I'm not saying never use a parser generator, but I would personally prefer to work on a project with a well tested hand-written parser than a project using a parser generator. Especially if it complicates the build process with extra tooling, or is something really dated like Bison or ANTLR.
It is a culture thing. Try to avoid cowboy shops. The thing is the general standard seems higher than 20 years ago. Source control, unit testing and CI/CD are not controversial any more for example.
Yep. If you have written production grade software at real companies, you know that the moment you make that new commit (even if 1 liner change), you are now ready to accept that it could break something. yes you can do your unit tests, integration test, User Acceptance Tests and what not. But every code change = new possible bug that you may not be able to catch until it occurs to a customer.
Whenever I hear a developer say "I never ship buggy code", I am always cautious to dig in more and understand what they mean by that.
It's always amazing when I get a bug report from a product that's been running bug free in production for years with minimal changes but some user did some combination of things that had never been done and it blows up.
Usually it's something extremely simple to fix too.
This happens a lot more than one may think especially with products that have lot of features. Some features are used sparingly and the moment a customer uses that feature a bit more in depth, boom. Something is broken.
> especially with products that have lot of features
No kidding. I'm 2 or 3 years into working on a SaaS app started in ~2013 and I still get bug reports from users that make me say "what!? we have that feature!?"
I never really got how proofs are supposed to solve this issue. I think that would just move the bugs from the code into the proof definition. Your code may do what the proof says, but how do you know what the proof says is what you actually want to happen?
A formal spec isn't just ordinary source-code by another name, it's at a quite different level of abstraction, and (hopefully) it will be proven that its invariants always hold. (This is a separate step from proving that the model corresponds to the ultimate deliverable of the formal development process, be that source-code or binary.)
Bugs in the formal spec aren't impossible, but use of formal methods doesn't prevent you from doing acceptance testing as well. In practice, there's a whole methodology at work, not just blind trust in the formal spec.
Software developed using formal methods is generally assured to be free of runtime errors at the level of the target language (divide-by-zero, dereferencing NULL, out-of-bounds array access, etc). This is a pretty significant advantage, and applies even if there's a bug in the spec.
> A formal spec isn't just ordinary source-code by another name, it's at a quite different level of abstraction
This is the fallacy people have when thinking they can "prove" anything useful with formal systems. Code is _already_ a kind formal specification of program behavior. For example `printf("Hello world");` is a specification of a program that prints hello world. And we already have an abundance of tooling for applying all kind of abstractions imaginable to code. Any success at "proving" correctness using formal methods can probably be transformed into a way to write programs that ensure correctness. For example, Rust has pretty much done so for a large class of bugs prevalent in C/C++.
The mathematician's wet dream of applying "mathematical proof" on computer code will not work. That said, the approach of inventing better abstractions and making it hard if not impossible for the programmer to write the wrong thing (as in Rust) is likely the way forward. I'd argue the Rust approach is in a very real way equivalent to a formal specification of program behavior that ensures the program does not have the various bugs that plagues C/C++.
Of course, as long as the programming language is Turing Complete you can't make it impossible for the programmer to mistakenly write something they didn't intend. No amount of formalism can prevent a programmer from writing `printf("hello word")` when they intended "hello world". Computers _already_ "do what I say", and "do what I mean" is impossible unless people invent a way for minds to telepathically transmit their intentions (by this point you'd have to wonder whether the intention is the conscious one or the subconscious ones).
> thinking they can "prove" anything useful with formal systems
As I already said in my reply to xmprt, formal methods have been used successfully in developing life-critical code, although it remains a tiny niche. (It's a lot of work, so it's only worth it for that kind of code.) Google should turn up some examples.
> Code is _already_ a kind formal specification of program behavior.
Not really. Few languages even have an unambiguous language-definition spec. The behaviour of C code may vary between different standards-compliant compilers/platforms, for example.
The SPARK Ada language, on the other hand, is unambiguous and is amenable to formal reasoning. That's by careful design, and it's pretty unique. It's also an extremely minimal language.
> `printf("Hello world");` is a specification of a program that prints hello world
There's more to the story even here. Reasoning precisely about printf isn't as trivial as it appears. It will attempt to print Hello world in a character-encoding determined by the compiler/platform, not by the C standard. It will fail if the stdout pipe is closed or if it runs into other trouble. Even a printf call has plenty of complexity we tend to just ignore in day to day programming, see https://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodb...
> Any success at "proving" correctness using formal methods can probably be transformed into a way to write programs that ensure correctness
You've roughly described SPARK Ada's higher 'assurance levels', where each function and procedure has not only an ordinary body, written in SPARK Ada, but also a formal specification.
SPARK is pretty challenging to use, and there can be practical limitations on what properties can be proved with today's provers, but still, it is already a reality.
> Rust has pretty much done so for a large class of bugs prevalent in C/C++
Most modern languages improve upon the appalling lack of safety in C and C++. You're right that Rust (in particular the Safe Rust subset) does a much better job than most, and is showing a lot of success in its safety features. Programs written in Safe Rust don't have memory safety bugs, which is a tremendous improvement on C and C++, and it manages this without a garbage collector. Rust doesn't really lend itself to formal reasoning though, it doesn't even have a proper language spec.
> The mathematician's wet dream of applying "mathematical proof" on computer code will not work
Again, formal methods aren't hypothetical.
> I'd argue the Rust approach is in a very real way equivalent to a formal specification of program behavior that ensures the program does not have the various bugs that plagues C/C++.
It is not. Safe languages offer rock-solid guarantees that certain kinds of bugs can't occur, yes, and that's very powerful, but is not equivalent to full formal verification.
It's great to eliminate whole classes of bugs relating to initialization, concurrency, types, and object lifetime. That doesn't verify the specific behaviour of the program, though.
> No amount of formalism can prevent a programmer from writing `printf("hello word")` when they intended "hello world"
That comes down to the question of how do you get the model right? See the first PDF I linked above. The software development process won't blindly trust the model. Bugs in the model are possible but it seems like in practice it's uncommon for them to go unnoticed for long, and they are not a showstopper for using formal methods to develop ultra-low-defect software in practice.
> "do what I mean" is impossible unless people invent a way for minds to telepathically transmit their intention
It's not clear what your point is here. No software development methodology can operate without a team that understands the requirements, and has the necessary contact with the requirements-setting customer, and domain experts, etc.
I suggest taking a look at both the PDFs I linked above, by way of an introduction to what formal methods are and how they can be used. (The Formal methods article on Wikipedia is regrettably rather dry.)
I think the reason that formal proofs haven't really caught on is because it's just adding more complexity and stuff to maintain. The list of things that need to be maintained just keeps growing: code, tests, deployment tooling, configs, environments, etc. And now add a formal proof onto that. If the user changes their requirements then the proof needs to change. A lot of code changes will probably necessitate a proof change as well. And it doesn't even eliminate bugs because the formal proof could include a bug too. I suppose it could help in trivial cases like sanity checking that a value isn't null or that a lock is only held by a single thread but it seems like a lot of those checks are already integrated in build tooling in one way or another.
Yes, with the current state of the art, adopting formal methods means adopting a radically different approach to software development. For 'rapid application development' work, it isn't going to be a good choice. It's only a real consideration if you're serious about developing ultra-low-defect software (to use a term from the AdaCore folks).
> it doesn't even eliminate bugs because the formal proof could include a bug too
This is rather dismissive. Formal methods have been successfully used in various life-critical software systems, such as medical equipment and avionics.
As I said above, formal methods can eliminate all 'runtime errors' (like out-of-bounds array access), and there's a lot of power in formally guaranteeing that the model's invariants are never broken.
> I suppose it could help in trivial cases like sanity checking that a value isn't null or that a lock is only held by a single thread
No, this doesn't accurately reflect how formal methods work. I suggest taking a look at the PDFs I linked above. For one thing, formal modelling is not done using a programming language.
You mix up development problem with computational problem.
If you can't use formal proof just because the user can't be arsed to wait where it is supposed to be necessary, then the software project conception is simply not well designed.
As always, the branding of formal methods sucks. As other commentators point out, it isn't technically possible to provide a formal proof that software is correct. And that is fine, because formal software methods don't do that.
But right from the outset the approach is doomed to fail because its proponents write like they don't know what they are talking about and think they can write bug-free software.
It really should be "write software with a formal spec". Once people start talking about "proof" in practice it sounds dishonest. It isn't possible to prove software and the focus really needs to be on the spec.
> It really should be "write software with a formal spec".
The code is already a formal spec.
Unless there are bugs in the language/compiler/interpreter, what the code is essentially formally well defined.
As programming languages get better at enabling programmers to communicate intention as opposed to being a way to generate computer instructions, there's really no need for a separate "spec". Any so called "spec" that is not a programming language is likely not "formal" in the sense that the behavior is unambiguously well defined.
Of course, you might be able to write the "spec" using a formal language that cannot be transformed into machine code, but assuming that the "spec" is actually well defined, then it's just that "compiling" the spec into machine code is too expensive in some way (eg. nobody has written a compiler, it's too computationally hard to deduce the actual intention even though it's well defined, etc.). But in essence it is still a "programming language", just one without a compiler/interpreter.
You can formally prove that it doesn't have certain kinds of bugs. And that's good! But it also is an enormous amount of work. And so, even for life-critical software, the vast majority is not formally proven, because we want more software than we can afford to formally prove.
This is an interesting point that I think a lot of programming can miss.
Proving that the program has no bugs is akin to proving that the program won't make you feel sad. Like ... I'm not sure we have the math.
One of the more important jobs of the software engineer is to look deep into your customer's dreams and determine how those dreams will ultimately make your customer sad unless there's some sort of intervention before you finish the implementation.
Exactly, it's fundamentally impossible. Formal proofs can help with parts of the process, but it can guarantee no bugs in the product. These are the steps of software, and their transitions. It's fundamentally a game of telephone with errors at each step along the way.
What actually would solve the customer's problem -> What the customer thinks they want -> What they communicate that they want -> What the requirements collector hears -> What the requirements collector documents -> How the implementor interprets the requirements -> What the implementor designs/plans -> What the implementor implements.
Formal proofs can help with the last 3 steps. But again that's assuming the implementor can formalize every requirement they interpreted. And that's impossible as well, there will always be implicit assumptions about the running environment, performance, scale, the behavior of dependent processes/APIs.
It helps with a small set of possible problems. If those problems are mission-critical then absolutely tackle them, but there will never be a situation where it can help with the first 5 steps of the problem, or with the implicit items in the 6th step above.
Even formally proved code can have bugs. If your requirement is wrong is the obvious thing. I don't work with formal proofs (I want to, I just don't know how), but I'm given to understand they have other real world limits that make them sometimes have other bugs.
Or perhaps you'll prove it from first principles. Although if turns out to be difficult, that's okay. Somebody mentioned something about systems being either complete or consistent but never both. Some things can be true but not proveably so. Can't quite remember who it was though.
Gödel's really was a rather unique mind, and the story of his death is kind of sad.. but I wonder if it takes such a severe kind of paranoia to look for how math can break itself, especially during that time when all the greatest mathematicians were in pursuit of formalizing a complete and consistent mathematics.
No. It merely prevents you from confirming every arbitrarily complex proof. Incompleteness is more like: I give you a convoluted mess of spaghetti code and claim it computes prime numbers and I demand you try to prove me wrong.
There are well-formed statements that can be proved but which assert that its godelized value represents a non-provable theorem.
Therefore, you must accept that it and its contradiction are both provable (leading to an inconsistent system), or not accept it and now there are provable theorems that cannot be expressed in the system.
Furthermore, that this can be constructed from anything with base arithmetic and induction over first-order logic (Gödel's original paper included how broadly it could be applied to basically every logical system).
The important thing to note is that it doesn't have anything to do with truth or truth-values of propositions. It breaks the fundamental operation of the provability of a statement.
And, since many proofs are done by assuming a statement's inverse and trying to prove a contradiction, having a known contradiction in the set of provable statements can effectively allow any statement to be proven. Keeping the contradiction is not actually an option.
> If you believe you can ship bug free code, it's time to switch careers.
Unfortunately, you are correct. Shipping in time and bug free are inversely proportional, and in a world were usually it's hard to argue with PMs for more time to have better testing, or paying tech debt... it's just a reality
An infinite amount of time would not necessarily yield zero bugs.
But more importantly, once you've fixed the "show-stopping bugs," putting the software in front of customers is probably the best next step, as even if it's bug-free, that doesn't mean it solves the problem well.
there is no such thing as zero bugs. There is only a marker in time for a suite of tests that show no bugs. Doesn't mean larva aren't living under the wood. You can't control all the bits (unless you built your own hardware/software stack).
I think we're saying the same thing? That was my point. You're never going to achieve zero bugs no matter how much time you give yourself. Focus on getting the critical path right and creating a good experience, and then get it to customers for feedback on where to go next.
[The above does not necessarily apply in highly regulated industries or where lives are on the line]
I like to think of "zero bugs" as the asymptote. As you spend more time, you discover increasingly fewer (and less significant) bugs per unit of time. POSSIBLY at the limit of infinite time you hit 0 bugs, but even if you could, would it be worth it? Doubtful.
I can think of far better ways to spend infinite time.
0 bugs is actually impossible. A cosmic ray can flip a bit and change the behavior of your software. We live in a fundamentally unreliable universe.
We aren't taught how to write reliable software because very few people know how to write reliable software. It doesn't help that academia has a hard crush on OOP, which is a bag of accidental complexity - and complexity is a breeding ground for unreliability.
I think if a cosmic ray flips the bit and changes the behavior of your software, you can still reasonably brag that you wrote 0-bug code. It's not your fault that happened, you didn't do that. The code you wrote had 0 bugs.
I would say that also applies on highly regulated industries or where lives are on the line.
On those you're of course expected to do safety and testing up to the limit of the "value of a statistical life"s within the expected project impacts, but it still has time and budget limits.
The part I was suggesting does not apply is the statement "Focus on getting the critical path right and creating a good experience, and then get it to customers for feedback on where to go next."
Most software engineering is about making sure the happy path works well. When lives are on the line, you need to also plan to minimize the possible damage that can happen when things go wrong.
Yup, I also like how you call out "get it in-front of customers" as a step in the whole chain. Often sorely missed. Sometimes a bug to you, is a feature to them (gasp!)... so either make it a first class thing or train them on the correct path (while you fix the "bug").
Ok, I think we’ve gone too far. There absolutely is such thing as 0 bugs and sometimes code changes don’t have bugs. That is not to say it can be garunteed.
We need to define bug but if bug is anything a customer (internal or external) is not happy with that passes triage and you can’t throw it back in their face. Then zero bugs would be impossible with even infinite time.
That's only true up to a point. There are some quality assurance and control activities that are essentially "free" in that they actually allow for shipping faster by preventing rework. But that requires a high level of team maturity and process discipline, so some teams are simply incapable of doing it. And even in ideal circumstances it's impossible to ship defect free software (plus the endless discussions over whether particular issues are bugs or enhancement requests).
yeah, it's a spectrum. Clearly no one is expecting an app to be truly bug free if the underlying compiler itself has bugs. But how often do users truly run into compiler level bugs?
I think when the author says "bug free", it's from the user perspective. where bugs either need to go out of your way to trigger or are so esoteric it's impossible to think about hitting without that user themself knowing the code inside out. Games is definitely an industry where the quality of code has always dipped to a point where users can easily hit issues in normal use, and only gets worse as games get more complex. That's where it gets truly intolerable
There are tools that help, but you still need time to integrate those tools, learn how to use them, etc. If you are doing unit and integration tests, you need time to not only write those, but also actually plan your tests, and learn how to write tests. Which... needs time
This kind of wisdom only comes from experience I think. Either that or higher order think. Like the article says, most of the time testing/tdd/qa is bolt on after-the-fact. Or a big push at the end with "QA Sprints" (are you sprinting or are you examining? what exactly is a QA sprint? I know what it is).
Once you get beyond "I wrote a function" and "I tested a function" and even still "I tested a function that was called by a function over the wire", you will come to a realization that no matter how edgy your edge cases, no matter how thorough your QA, there will always - ALWAYS be 0-day "undefined behavior" in certain configurations. On certain hardware. On certain kernels. It's an assurance that I guarantee that I'm almost positive it's bug free, since it passed tests, it passed human eyes, and it passed review - fingers crossed.
My wife works as an acoustical consultant at a global construction firm. The things you hear about factories, offices, and even hospitals is wild. Don’t get me wrong the construction world works very hard to avoid issues but I think we in software tend to hold other engineering disciplines up on a pedestal that doesn’t quite match the messiness of reality.
Thanks for saying this. I think we in software engineering tend to think too binary: either the product is perfect (100% bug-free) or it's shit. There's always room for improvement, but compared to other engineering, overall, I think we're doing pretty good. As an example similar to your wife's, my friend used to work for one of the major car manufacturers doing almost the exact same job as Edward Norton's character in Fight Club. The cars had "bugs", they knew about it, but they didn't publicly acknowledge it until they were forced to.
There are a few aspects. One is that we don't understand the fundamentals of software as well as the underpinnings of other engineering disciplines.
More importantly though, for the most part we choose not to do engineering. By which I mean this - we know how to do this better, and we apply those techniques in areas where the consequences of failure are high. Aerospace, medical devices, etc.
It differs a bit industry to industry, but overall the lessons are the same. On the whole it a) looks a lot more like "typical" engineering than most software development and b) it is more expensive and slower.
Overall, we seem to have collectively decided we are fine with flakier software that delivers new and more complex things faster, except where errors tend to kill people or expensive machines without intending to.
The other contributing thing is it's typically vastly cheaper to fix software errors after the fact than, say, bridges.
> One is that we don't understand the fundamentals of software as well as the underpinnings of other engineering disciplines.
That sounds like an awfully bold claim. I have the feeling we understand software a lot better than we understand mechanical engineering (and by extension material sciences) or fluid dynamics. By a big margin.
I worked with finite element software and with CFD solvers, you wouldn't believe how hard it is to simulate a proper airflow over a simple airfoil and get the same results as in the wind tunnel.
To the contrary, it's nearly canonical. Most of the problems pointed out in the 70s (mythical man month) have still not been resolved, 50 years later.
>you wouldn't believe how hard it
Oh, I'd believe it (I've designed and built similar things, and had colleagues in CFD).
But you are definitely cherry picking here. The problem with CFD is we don't understand the fluid dynamics part very well; turbulence is a big unsolved problem still, though we have been generating better techniques. This is so true that in an undergraduate physics degree, there is usually a point where they say something like: "now that you think you know how lots of things work, let's introduce turbulence"
But a lot of mechanical engineering and the underlying physics and materials science is actually pretty well understood, to the degree that we can be much more predictive about the trade offs than typically is possible in software. Same goes for electrical, civil, and chem. Each of them have areas of fuzziness, but also a pretty solid core.
> To the contrary, it's nearly canonical. Most of the problems pointed out in the 70s (mythical man month) have still not been resolved, 50 years later.
Even with all of those applied, we wouldn’t be magically better. Complexity is simply unbounded. It’s almost impossible to reason about parallel code with shared mutable state.
The article is about delivering a complete, working project "on time". I have a neighbor whose home is being renovated and it is already 2x the time the contractor originally quoted.
Of course it is easier for a developer to walk away from something incomplete than an architect and the contractors involved in a physical project, but still, I hardly think that there is really much difference in terms of timelines.
FWIW in my experience delays in e.g. home renos (or for that matter larger scale projects) are mostly for reasons unrelated to the engineering. In software projects, it's probably the #1 reason (i.e. we didn't know how to do it when we started).
Software is still absolutely king for number of large scale projects that just never ship, or ship but never work.
I think your salary observation is more of a firmware vs. hardware, rather then "soft" vs "hard" engineering.
Further to that, it's often informative to figure out what makes a company money. The highest paid software development roles tend to be doing things that are closer to revenue, on average. If you are a software developer at a hardware company (or an insurance company, or whatever), you aren't that close. Even worse if you are viewed as a cost center.
>Further to that, it's often informative to figure out what makes a company money. The highest paid software development roles tend to be doing things that are closer to revenue, on average.
yeah. Who are those trillion dollar businesses and what do they rely on?
- Apple: Probably the better example here since they focus a lot on user-facing value. But I'm sure they have their own deals, B2B market in certain industries, R&D, and ads to take into account
- Microsoft: a dominant software house in nearly every aspect of the industry. But I wager most of their money comes not from users but other businesses. Virtually every other companies uses Windows, Word, and those that don't may still use Azure for servers.
- Alphabet: ads. Need I say more? Users aren't the audience, they are the selling point to other companies.
- Amazon: a big user facing market, but again similar to Microsoft. The real money is b2b servers.
- Nvidia: Again, user facing products but the real selling point is to companies that need their hardware. In this case, a good 80% of general computing manufacturers.
- Meta: Ads ans selling user data once again
- Tesla: CEO politics aside, it's probably the 2nd best example. Split bewteen a user facing product that disrupted an industry and becoming a standard for fuel in the industry they disrupted. There's also some tangential products that shouldn't be underestimated, but overall a lot of value seems to come from serving the user.
General lesson here is that b2b and ads are the real money makers. if you're one level removed that financial value drops immensely (but not necessarily to infeasible levels, far from it).
Trust me when I say this:
even "other" engineering domains have to do patches.
The difference is that software can be used before it is fully ready, and it makes sense to do so. No one can really use a 90% finished power plant, but software at 95% capacity is still usually "good enough"
I think you're 90% there. There is also the cost to apply a patch.
If you want to patch a bridge, it's gonna cost you. Even if you only need to close down a single lane of traffic for a few hours you are looking at massive expenses for traffic control, coordination with transportation agencies, etc.
For most software it's pretty inexpensive to ship updates. If you're a SaaS company regular updates are just part of your business model. So the software is never actually done. We just keep patching and patching.
In some contexts, it is much more expensive to push out updates. For example, in the 00s, I worked on a project that had weather sensors installed in remote locations in various countries and the only way to get new software to them was via dial-up. And we were luck that that was even an option. Making international long distance calls to upload software patches over a 9600 baud connection is expensive. So we tested our code religiously before even considering an update, and we only pushed out the most direly needed patches.
Working on SaaS these days and the approach is "roll forward through bugs". It just makes more economic sense with the cost structures in this business.
Thanks for this insight! It has pretty strong explanatory power. It also explains why rushed development can stall. It explains 'move fast and break things'.
There's even an added factor of learning more about what is really needed by putting a 95% done product into use.
Heck, it explains (stretching it here) space-x's success with an iterative approach to rocket design.
I install high voltage switchgear on site.
A common problem is all the changes that has been added during the design stage, circuits that have been removed or altered, work that has kind of mostly been done to the schemes by the overworked secondary engineer. Sometimes, the schemes have been changed after all the wiring is completed and shipped to site, making it my pain in the ass when it's time to do the commissioning.
The end result is never 100% perfect, but somewhere in between "not too bad" and "good enough".
Team are flying the airplane, the se time rebuild it to the zeppelin, testing new engines inflight.
Or construction. Let's build apartment block, but for few apartments we will test new materials, new layout, etc. Once there are walls of the first apartments we will let people live there. We will build how we can, according to the napkin plan. In the end we will put all tenants in and stress test strength of the structures. Or one day people return home and their apartments have totally different design and layout because someone from the HOA decided so to get a promotion.
>when do not accept the same in other engineering domains?
No, you just complain that your taxes are being used to build expensive roads and bridges. Or you think airplanes are far too expensive. Or that new cars are insanely expensive.
There are cost trade offs. In general, better quality more expense.
Also in software there is not an excessive amount of software engineers in relation to demand for software. So SWEs can get paid a lot to go build crappy software.
Because complexity is boundless and in software it has no cost.
Building a house will have a restrictive initial budget for complexity, you don’t have enough in that budget for rotating floors, or an elevator that is catapulted to the correct floor, etc. These would cost both at engineering time and implementation time a huge amount. Less complexity is easier to analyze.
In case of software, complexity has negligible cost, relative to physical systems. You can increase it ad infinity — but proving it (the whole stack - from the hardware-OS-userspace software) correct is likely impossible with even the whole of mathematics, in certain cases.
In addition to the other answers, there is the perennial and depressing one: Software bugs haven't killed enough people in a suitably visible/dramatic way to be regulated that heavily.
We accept this in all fields of engineering. Everything is "good enough" and the seems to work reasonably well. You should remember this next time you hear about car recalls, maintenance work on bridges, or when some component in your laptop flakes out.
I mean, bridges collapse. That hasn't meant we gave up on engineering bridges.
Point being, we have some risk tolerance, even for civil engineering.
Now we don't accept an engineer saying, "this bridge will probably collapse without warning", which we do accept with software. So there is a difference.
It's perfectly acceptable to let bugs escape into production if those "cost" of fixing that bug higher than the "cost" to the user experience / job to be done. A bug that takes a week to fix that will only be encountered by a small amount of users in a small number of obscure scenarios may not need to be fixed.
I think a common error is taking this view in isolation on each bug.
Fact is, if you ship enough 'low probability' bugs in your product, your probabilities still add up to a point where many customers are going to hit several of them.
I've used plenty of products that suffer from 'death by a thousands cuts'. Are the bugs I hit "ship blockers"? No. Do I hit enough of them that the product sucks and I don't want to use it? Absolutely.
Very much this, and low risk bugs compound at scale.
If you're in a very large FANNG type company, and say you have 1000 components that each ship 1 bug each day that has a 0.1% chance of breaking something critical, that translates to a less than 50% chance you ship a working OS on any given day. And that may mean the entire company's productivity is impacted for the day depending on how broken it is.
Software is commonly built on non-fungible components and monopolies.
Right, you don't want to use Microsoft Word, or SalesForce, or Apple vs Android, or X Whatever. It's highly unlikely you'll have a choice if you use it though.
This presupposes that you know what/where bugs will be found and how they'll impact future users. In my experience knowing either of these is very rare at the point where you're "building quality".
it will be necessary to deliver software without bugs that could have reasonably been avoided in time
ive had this sentiment thrown at me too often by peak move fast and break things types. it's too often a cudgel to dispense with all QA in favor of more new feature development. shipping shit that has the same pattern of flaws youve encountered in the past when youve been shown ways to catch them early but couldnt be bothered isnt accepting that you cant catch everything, it's creating a negative externality.
you usually can make it someone else's problem and abscond with the profits despite, but that doesn't mean you should
I think with formal analysis, whole bug classes can be eliminated. Add to that a rigorous programming style, and 'bug-free' code is within reach.
There will remain bugss that make it through, but they will be rare, and will need a chain of mistakes.
Currently ways of coding to this kind of standard exist. But they are stupid. It involves things like no dynamic memory allocation, only fixed length for-loops, and other very strict rules. These are used in aerospace, where bugs are rather costly and rare.
What seems to yield much better results is to have the program be built by two separate teams to a standard. Then both programs can be run simultaneously, checking each others output — I believe something like this is actually used in the aerospace industry.
It is sad that people on here would believe this and that for whole platforms it is actually true, however, it absolutely is not universally true and the proof is all around us.
Yes, but that software is not bug-free. The claim was not "it's impossible to make software that does not exhibit bugs too a casually noticeable degree".
People who know how the sausage is made will always know of a bunch of bugs that haven't been fixed exactly because they aren't impactful enough to be worth the effort required to fix them.
If it works within specs it is bug free. It doesn’t matter how it is made if it works within specs, which is one of the real unfortunate truths of software.
The other is working out the correct specification is far harder than coding is.
For example it is trivial to write a bug free program that multiplies an integer between 3 and 45 by two.
Most devices work within the spec 99.9% of the time, but that last .1% it is outside the spec. The exact % is different for different projects of course, but the idea is still there: no software operates according to spec 100% of the time.
Sure, but adding two ints is trivial. Hello world probably operates to spec all the time too. Almost all software is vastly more compelx and isn't perfect.
Some people obviously aren't true Scotsman... I'm from the US and have no attachment to Scotland; if I claimed to be a Scotsman and you pointed out that I'm not, and I said "well that's just the no true Scotsman fallacy!", then I would be totally full of it.
In the same way I am not a real Scotsman, your toy example of an easily specified snippet of a function that doesn't do anything useful is not real software.
As you alluded, in practice no specs fully specify a truly bug free implementation. If you want to consider bugs that are within the specification as being bugs in the spec rather than bugs in the implementation, fine, but in my view that is a distinction without a difference.
(Personally, I think code is itself more analogous to the specification artifacts of other professions - eg. blueprints - and the process of creating the machine code of what is analogous to construction / manufacturing something to those specs.)
And even having said that, even the vast majority "bug free" software that nearly always appears to be operating "within spec" will have corner cases that are expressed in very rare situations.
But none of this is an argument for nihilism about quality! It is just not the right expectation going into a career in software that you'll be able to make things that are truly perfect. I have seen many people struggle with that expectation mismatch and get lost down rabbit holes of analysis paralysis and overengineering because of it.
> in practice no specs fully specify a truly bug free implementation.
Except for ones that do, obviously.
The key reason to make the distinction is because the fuzzy business of translating intention into specification needs to be fully accepted as an ongoing negotiation process of defining exactly what the specification is, and integrated into repeated deterministic verification that that is what has been delivered. Failing to do that is mainly a great way for certain software management structures to manipulate people by ensuring everything is negotiable all the time, and has the side effect that no one can even say if something is a bug or not. (And this pattern is very clear in the discussion in this thread - there is a definite unwillingness to define what a bug is).
IME the process of automated fuzzing radically improves all round quality simply because it shakes out so many of the implicit assumptions and forces you specify the exact expected results. The simple truth is most people are too lazy and/or lack the discipline needed to do it.
Those don't exist. There are too many free variables. Some get much closer than others (for instance via formal verification), but all specs are by necessity a model of reality, not the full reality itself.
Nobody actually has any trouble knowing what a bug is. Like, this is just a non-issue, I've never in my career spent a non-negligible amount of time debating with anybody whether something is or isn't a bug. We discuss whether fixing bugs have worthwhile return on investment, and we discuss the relative priority between fixing bugs and doing other things, but this meta-debate about "well technically it complies with this spec so is it even a bug, really?" just never comes up. We all know what bugs are.
Fair enough. I considered a bug to be any behavior the engineers didn't plan in the code. They have their own specification, in their heads, that is more technical/exact than the business specification. Your definition is also reasonable but it's not what people mean when they say "there's no such thing as bug-free code", because bugs of my definition are almost unavoidable.
To have no bugs, which is extremely unlikely for a program of any real complexity. Having bugs, and being functional, are fairly self-explanatory and independent of each other. No need to try to conflate them.
Not sure what your quote is supposed to mean. That's a textbook example of someone who doesn't understand software at all making laughable requests of their engineers.
To be bug free we must be able to define what a bug is. So, what is a bug?
The reason for that quote is from what you have said a bug would be anything you didn't expect, even if it is consistent or not with the specification as that merely affects if we classify it as functional or not (a classification I profoundly disagree with, obviously). It is simply a negative rephrasing of what the marketing guy said and laughable in the same way.
> One plausible definition is “system deviates from its specification”
And that's quite reasonable. So I actually retract my argument.
For my own definition, I was considering a bug to be any behavior that the software engineers weren't expecting. Because those can exist invisibly for a long time until they become so bad they become visible. They can also exist for decades without causing any problems to functionality at all.
Not buildings, not bridges, not cars, not airplanes, not software. There are mistakes in every field of engineering and the best we can hope for is to minimize them as much as possible.
Engineering is knowing (among other things) how to categorize the potential mistakes, develop procedures to reduce the chance of them being made and in the case that some slip through (and they will), estimate their impact and decide when you're "good enough."
Reminds me of a story about an engineer who was participating in a meeting with managers and partners. Manager was speaking of his team and how they will deliver software. Then, he asked the engineer to assure that the software will be bug-free. To this the engineer responded by saying he cannot guarantee there will be no bugs. The manager went nuts and started screaming.
Engineers cannot be responsible for all the vertical stack and the components which were built by others. If somebody claims it is bug free then they have not enough experience. Pretty much anything can fail, we just need to test as many possible cases as possible with a variety of available tools to reduce the chances of bugs.
You might be correct today but that’s a pretty sad state of affairs, don’t you think we can do better? Most other engineering domains can deliver projects without bugs, with various definitions of “bug” of course
I'm not sure about that. Which engineering domain do you have in mind?
Maybe show-stopping bugs are somewhat unique to software engineering, but all somewhat-complex products are flawed to some extent imho.
It might be an unergonomic handle on a frying pan, furniture that visibly warps under the slightest load (looking at ikea shelfing) or the lettering coming off the frequently used buttons on a coffee machine.
But there do exist shelves that don’t warp, when used within some reasonable bounds.
I’d also quibble about the buttons on the coffee machine. They might be properly designed, just subject to the normal wear-and-tear that is inevitable in the real world. This is not a defect, physical devices have finite lifespans.
As far as computers go… if we got to the point where the main thing that killed our programs was the hard drives falling apart and capacitors drying out, that would be quite impressive and I think everyone would be a little bit less critical of the field.
Formally verified, bug free software exists. It just costs a LOT to produce, and typically isn't worth it, except for things like cryptographic libraries and life or death systems.
As the discipline has evolved, the high integrity tools are slowly being incorporated into typical languages and IDEs to generally improve quality cheaper. Compare C++ to rust for example, whole classes of bugs are impossible (or much harder to make) in rust.
A shelve is a dumb primitive static object though. Even a simple hello world goes over a huge amount of lines of code before it is displayed on a screen, ANY one of which being faulty could result in a bug visible to the enduser. And most of that is not even controlled by the programmer — they might call into libc, which calls into the OS, which calls into drawing/font rendering libraries, that calls into video card drivers that “calls” into the screen’s firmware.
I think “hello world” is not really the simplest program in this context, in the sense that printing, as you note, involves touching all that complicated OS stuff. In terms of, like, actual logic complexity implemented by the programmer compared to mess carried along by the stack, it is really bad.
But I mean, I basically agree that the ecosystem is too complicated.
To be an engineer is to know the expected system requirements and build a product that is extremely optimized for the system requirements.
There's a saying that I think fits very well here: "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."
You don't want a bridge to cost 50 years and quadrillions of dollars to build, you want a cheap bridge safe for the next 50 years done in 2 years.
I would not call the resulting bridge "bug free", of course.
We can certainly do better, but it takes a _lot_ of time, effort, care and discipline; something most teams don't have, and most projects can't afford.
Bugs arise from the inherent complexity introduced by writing code, and our inability to foresee all the logical paths a machine can take. If we're disciplined, we write more code to test the scenarios we can think of, which is an extremely arduous process, that even with the most thorough testing practices (e.g. SQLite) still can't produce failproof software. This is partly because, while we can control our own software to a certain degree, we have no control over the inputs it receives and all of its combinations, nor over the environment it runs in, which is also built by other humans, and has its own set of bugs. The fact modern computing works at all is nothing short of remarkable.
But I'm optimistic about AI doing much better. Not the general pattern matching models we use today, though these are still helpful with chore tasks, as a reference tool, and will continue to improve in ways that help us write less bugs, with less effort. But eventually, AI will be able to evaluate all possible branches of execution, and arrive at the solution with the least probability of failing. Once it also controls the environment the software runs in and its inputs, it will be able to modify all of these variables to produce the desired outcome. There won't be a large demand for human-written software once this happens. We might even ban software by humans from being used in critical environments, just like we'll ban humans from driving cars on public roads. We'll probably find the lower quality and bugs amusing and charming, so there will be some demand for this type of software, but it will be written by hobbyists and enjoyed by a niche audience.
A saying that I once heard and appreciate goes like this:
"A programmer who releases buggy software and fixes them is better than a programmer who always releases perfect software in one shot, because the latter doesn't know how to fix bugs."
Perhaps similar to the saying that a good driver will miss a turn, but a bad driver never misses one.
I think you misunderstand, I'm talking about a programmer who makes perfect, bug-free code in one shot. There are no bugs to catch and fix, because this "perfect" programmer never writes buggy code.
The moral of the sayings is, that "perfect" programmer is actually a bad programmer because he wouldn't know how to fix bugs by virtue of never needing to deal with them.
To reuse the driver analogy, the driver who never misses a turn is a bad driver because he doesn't know what to do when he does miss a turn.
If a software developer consistently delivers high-quality software on time and on budget, that means they're good at their job, pretty much by definition. It would make no sense to infer they're bad at fixing bugs.
It would make sense to infer instead that they're good at catching and fixing bugs prior to release, which is what we want from a software development process.
> the driver who never misses a turn is a bad driver because he doesn't know what to do when he does miss a turn
Missing a turn during a driving test will never improve your odds of passing.
The driver who never misses a turn presumably has excellent awareness and will be well equipped to deal with a mistake should they make one. They also probably got that way by missing plenty of turns when they were less experienced.
What we are discussing isn't a real programmer we might actually find. No, we are talking about a hypothetical "perfect" programmer. This "perfect" programmer never wrote a bug in his entire life right from the moment he was born, he never had a "when they were less experienced" phase.
Obviously, that means this "perfect" programmer also never debugged anything. For all the perfect code he writes, that makes him worse than a programmer who writes buggy code but also knows how to go about debugging them.
I get that this is a blog post and needfully short, but yes, there are courses that teach these skills. There's a big disconnect between CS and SE in general, but it's not as bad as "no one teaches how to build quality software". We do work on this.