This blog is no longer active, you can find my new stuff at: george3d6.com, epistem.ink, ontologi.cc, and phenomenologi.cc
Exceptions as control flow
What do I mean by using exceptions as a control flow mechanism?
I - Exceptions as control flow mechanism
The gist of it is replacing something like:
if isinstance(a, duck):
a.quack()
else:
bar()
With something like:
try:
a.quack()
except:
bar()
Writing control flows using an if/switch/etc involves thinking about something like:
Unde what conditions should I perform this operation and what should I do if those conditions aren't true?
Writing control flows using exception switched that paradigm to:
If this operation fails, what should I try instead?
An example that I encountered in the real world:
As part of a data cleaning script, I had to determine if a float different from NaN in python. But really, what I wanted to ask was: "Can I use this variable of type float/np.float/whatever in basic numerical operations without raising exceptions or running into undefined behavior ?".
The obvious way to do that is something like: math.isnan(number), right?
Well, technically correct, as long as your definition of NaN is the same as the one python has. But under my definition, this will fail in cases like the following: np.isnan(float('inf')).
So I have to write a function like:
def is_invalid_float(nr):
if np.isnan(nr):
return True
elif np.isinf(nr):
return True
else:
return False
This has some disadvantages:
a) I have to know about the fact that a floating-point variable can be both nan and inf.
b) I may still be missing some edge cases in which a float typed variable can lead to UB or exception throwing in numeric operations.
c) It's a somewhat complex function, instead of a one-liner calling an external function.
The exception-based version looks something like:
def is_invalid_float(nr):
try:
int(nr)
return False
except:
return True
Much better, right?
Well, arguably. Going through my complaints above:
a) I have to know about the fact that a floating-point variable can be both nan and inf. -- I don't have to know this, if I implemented my nan check like this to being with, I would have been safe and sound never knowing about inf.
b) I may still be missing some edge cases in which a float typed variable can lead to UB or exception throwing in numeric operations. -- Well, I might still be missing some edge cases, but there's a higher chance I won't be, since I can assume other "strange" things that may be of type float respect the rule 'if it's a non-numeric floating-point typed variable I can't make an integer out of it".
c) It's still a somewhat complex function. Shorter, but it's less obvious what kind of values (nan and inf) caused the need for this check.
More importantly, I'm losing potentially valid floats that can't be converted to integers for some other reasons.
Note: as far as I know there are no other "edge cases" for floating-point numbers in python other than nan and inf and any other floating-point numbers will be int-convertible on an x86 CPython RE. This is an example for illustrative purposes
Ok, let's look at a stronger example:
Let's say I have a string and I need to check it's a read-only MariaDB query, something that only selects data rather than inserting, modifying, or deleting it.
Well, I can use an external SQL parser to tell me if the query is valid and contains mutable keywords or I can try to check for them myself, or I can:
def is_read_only_query():
with new_connection() as conn:
try:
conn.execute('SET SESSION TRANSACTION READ ONLY')
conn.execute(query_string)
return True
except:
return False
Is this a bullet-proof solution?
I don't know, I wouldn't use it to prevent a SQL injection from a skilled and malicious actor. But it can avoid a "Robert); DROP TABLE Students;" kind of scenario and it can certainly avoid a mistake by a clumsy database operator with the best of intentions but a habit of too readily copy-pasting stuff from stack overflow.
What I'm saying here is not revolutionary, at least not for certain languages, catching an exception is often the recommended way to check if an object can be cast to a different type in python.
I think where people would get a bit more itchy is when seeing something like:
def determine_type(text):
try:
sum(list(text))
return 'list of numbers'
except:
try:
int(text)
return 'int'
except:
try:
float(text)
return 'float'
except:
return 'other'
To me, the above code looks like the "correct" way to check if text is a list of numbers, integer, float, or a string representing something else.
But I can understand the reluctance to use these kinds of control flows. They are really ugly in terms of brackets or indentation, so getting past 6 or 7 layers of this can get unreadable. There is no easy workaround for this in dialects that people are familiar with.
But, what if we keep it short and sweet? What's the disadvantage of exception-based control flow?
II - Stack Overflow on why it's bad
I'm curious what the answer is if I google "Exceptions as control flow". I tried a few queries, this one returned varied results which bought up good points, so I stuck with it, but this is not an exhaustive list of arguments against the practice.
First, let's look at some of the most upvoted answers (in order of votes) on the Stack Overflow question on the subject:
Have you ever tried to debug a program raising five exceptions per second in the normal course of operation ?
I have.
The program was quite complex (it was a distributed calculation server), and a slight modification at one side of the program could easily break something in a completely different place.
I wish I could just have launched the program and wait for exceptions to occur, but there were around 200 exceptions during the start-up in the normal course of operations
My point: if you use exceptions for normal situations, how do you locate unusual (ie _exception_al) situations?
Of course, there are other strong reasons not to use exceptions too much, especially performance-wise
I think the performance bit is very niche, in code that gets execute thousands of times per second in a high-performance application it matters. But "programming" becomes very different when focusing on performance (e.g. we might write logic that looks counter-intuitive to us but results in fewer cache misses), and for 99% of code, this doesn't matter.
Granted, there's some language in which exceptions are really (and I mean *** really) slow, but others, such as Python, have fairly efficient exceptions so performance shouldn't be an issue for any program that should have been written in Python to being with.
The first point the commenter brings up sounds much more persuasive. Namely that exceptions, even when caught, are something one should care about.
I.e. my code might throw an exception during runtime and I'll catch it and try to handle the situation and move forward as best I can, but I need to see those exceptions when debugging.
This is a valid argument but there are two ways we can deal with this:
a) Add some code to indicate a specific catch is not meant to occur
b) Have specific exceptions for situations that should not happen (i.e. an "TheFactThatThisHappenedIsBadPleasePayAttention" exception class)
But still, this means that you'd have to potentially redesign your codebase a bit if you wanted to use exceptions as a control flow mechanism and still make use of their power to signal potential errors without crashing the whole program to do so.
The second answer in that same thread is:
Exceptions are basically non-local goto statements with all the consequences of the latter. Using exceptions for flow control violates a principle of least astonishment, make programs hard to read (remember that programs are written for programmers first).
Moreover, this is not what compiler vendors expect. They expect exceptions to be thrown rarely, and they usually let the throw code be quite inefficient. Throwing exceptions is one of the most expensive operations in .NET.
However, some languages (notably Python) use exceptions as flow-control constructs. For example, iterators raise a StopIteration exception if there are no further items. Even standard language constructs (such as for) rely on this.
Again the performance argument, but at least with the added caveat that indeed in some languages this is a none issue.
The more interesting point is that exceptions are harder to reason about since you're not sure where they are thrown, take for example the following code:
try:
a()
b()
c()
except:
foo()
When calling foo here we can't know why foo was called (i.e. was it because of a failure in a, b or c ?).
If it's non-obvious where an exception might be thrown one can simply implement multiple exception catching constructs (in the above case one for each function that is called).
But then again, I think this argument doesn't hold much water when we consider something like:
if not a() or not b() or not c():
foo()
Not knowing what code triggered an exception because multiple bits of it can throw is the same as not knowing what code triggered an if because multiple bits of it can set the True/False value that it evaluates.
This isn't a fault with exceptions, just a general issue one can introduce using any control flow mechanism.
Other than that, necromancer makes an interesting point in that thread, but most other arguments seem to follow the same ideas but phrased more poorly.
III - Stack Exchange SE on why exceptions are bad
Next, let's look at a fairly popular post on the software engineering stack exchange.
Why are exceptions bad?
Well, blueberryfields makes a few arguments for why they should be considered an anti-pattern. Let's take these one by one:
Exceptions are, in essence, sophisticated GOTO statements
They aren't, they have stricter rules than goto statements. Yes, they are a sub-case of goto but so is literally any other control structure. This seems to be a (very poor) argument of guilt-by-association.
Programming with exceptions, therefore, leads to more difficult to read, and understand code
Again, I think this is missing the fundamental point of when exceptions should be used and what their advantage is. I outline this above, but, in short:
- Programming with exceptions lead to messier code when one knows when something should fail/succeed to execute.
- Programming with exceptions leads to less messy code when one doesn't know when something should fail/succeed to execute, just what to try next if it fails.
Most languages have existing control structures designed to solve your problems without the use of exceptions
Arguably untrue (see argument above), but also, if you want to follow this line of thought one must next argue that we should remove all flavors of control flow other than if/else and while. If we are to pick between various control flows, I think exceptions are as valid a choice as switch in favor of else/if or for in favor of while.
Arguments for efficiency tend to be moot for modern compilers, which tend to optimize with the assumption that exceptions are not used for control flow.
Addressed previously.
I think Mason Wheeler makes a much better argument for when not to use exceptions.
The use case that exceptions were designed for is "I just encountered a situation that I cannot deal with properly at this point, because I don't have enough context to handle it, but the routine that called me (or something further up the call stack) ought to know how to handle it."
The secondary use case is "I just encountered a serious error, and right now getting out of this control flow to prevent data corruption or other damage is more important than trying to continue onward."
If you're not using exceptions for one of these two reasons, there's probably a better way to do it.
I pretty much agree with this point, but I think this actually leaves a lot of leeway for "exceptions as control flow".
A lot of libraries implement exceptions to cover the first case, but we are often not aware when those libraries will throw exceptions, just that they do.
Being unaware of when int throws an exception if applied to a float, I can still speculate "Hmh, this will probably happen if the float resides in a weird edge case such as nan", without having to know the logic of how to check for nan.
Furthermore, even the functions we write ourselves will often be in a situation where they don't (and shouldn't) have all the context to deal with an error, but enough context to know if it's an error. So even if we are to write a, b and c, the following construct could still be perfectly valid:
try:
res = a()
except:
try:
res =b()
except:
res = c()
Otherwise, we'd have to write something like: a(failure_func=partial(b,failure_func=c)) or make a and b return both a result and an error type (which still has to be interpreted in an if).
IV - C2 Wiki on why exceptions are bad
Finally, let's go to a more niche yet arguably much better source, C2. I'll be trying to address the main points in "Dont Use Exceptions For Flow Control". We know this is legit because the author doesn't bother with URL-unfriendly trappings of the English language such as '.
The article, really more of a collection of snippets, brings up most points mentioned above, not much new, and ends up being quite lukewarm on the issue. The only point it brings up that I haven't addressed is something like this:
it is company policy to work according to the following flow: First check if it can be done, if not, show an error, log, or whatever ..., if it can be done, do it. The major benefit is that we have a uniform way to do something; instead of using exceptions in one case and a "check" method in the other, we always use the "check" method. This discussion should be placed somewhere else, perhaps to CheckDontCatch. -- PeterDeBruycker
This boils down to the fact that using Exceptions in combination with if/else results in programmers being able to understand two patterns:
One where the original developers understood what situations the code ought to fail in and one where he didn't, he just understood that it could fail and that the failure could be handled.
This is a fairly strong point since in a large codebase if/else statements can inform new developers of how a given piece of logic works. Whereas with an exception, the original developer might have some incomplete intuitions as to when the exceptions will be thrown, though not complete enough to replace it with pan if/else, but the next developer will have access to none of those intuitions from reading the code.
I can't argue much here and must agree with the author, but I'm not sure how big of an issue this is provided a good comment practice is enforced. And, provided no such practice can be enforced, a practice that says "don't use exceptions as control flow" is also hard to enforce.
V - My own perspective
To add to the above, I think a large problem with using exceptions as control flow is that you may end up modifying the external state in ways that are unknown to you.
if sum([not isinstance(x,float) for x in arr]) == 0:
mutable_func_1(arr)
else:
mutable_func_2(arr)
This code is well defined unless the condition is meat mutable_func_1 never gets to modify the array.
But what about:
try:
for x in arr:
assert isinstance(x,float)
mutable_func_1(arr)
except:
mutable_func_2(arr)
Here it may well be that mutable_func_1 is the exception thrower and the arr object has already been modified when mutable_func_2 is called. Which might be less preferable than just crashing, what would have happened in the first case if mutable_func_1 threw.
So for one to use exceptions as a control flow mechanism one must make sure that ALL functions used inside the control flow are:
a) Not mutating any external state (outside of, maybe, throwing exceptions, if you want to view that as mutating external state).
b) Not throwing any exceptions and being called last in the control flow block, if they are mutating some external state.
This is much more strict than and if/else construct where one can go wild and use anything inside the individual blocks, with the assurance that there's no weird scenario where half of one block gets executed and than another block is executed.
This, in my opinion, is the biggest hurdle with using exceptions as control flow, and I'm actually surprised none of the articles/answers I found phrase the problem thusly.
VI - Conclusion
Exceptions are control-flow mechanism are unique in that they exchange the usual control-flow question of:
When should I be executing this code?
For:
Given that executing this code fails, what should I try next?
This is a potentially powerful exchange since one might often be able to easily answer one of these questions but not both.
However, there are a few caveats with this approach:
- The functions used inside an exception based control flow must be either non-mutable to external state or only execute mutations once we are certain no further exceptions can be thrown.
- Provided poor commenting, it's easier to lose track of the programmers original intent for when a certain block in the control flow should be triggered than with an if/else or switch construct.
- Exceptions can be very slow in certain languages and runtime environments compared to all other control flow mechanisms.
Overall this quick investigation biases me a bit more towards trying to use exceptions as control flows in situations where they suit, but I'm curious if there are still various downsides to the approach that I'm missing.
Published on: 2020-09-22
