Last updated on
Week 2: Higher-Order Functions and Debugging
Welcome to week 2 of CS-214 — Software Construction!
-
In the first part of this exercise set, we’ll revisit week 1’s exercises, and search for ways to leverage the power of higher-order functions to write shorter, simpler code. This first part is important for this week’s lab, and it will work much better if you work in groups.
-
In the second part of this exercise set, we’ll make ourselves familiar with the notion of “functions as values”. We’ll store functions in
vals, combine them, make functions that make functions that make functions, etc. We also encourage you to work in groups on this part. -
Finally, in the third part of this exercise set, we’ll train your debugging skills based on the SE lectures.
As before, ⭐️ indicates the most important exercises and questions and 🔥 indicates the most challenging ones. 🔜 indicates exercises that are useful to prepare for future lectures.
You do not need to complete all exercises to succeed in this class, and you do not need to do all exercises in the order they are written.
We strongly encourage you to solve the exercises on paper first, in groups. After completing a first draft on paper, you may want to check your solutions on your computer. To do so, you can pull from the course exercises repository.
Higher-Order Functions
This week, we’re trying something a bit different. At the front of the class you should find a stack of printed index cards (also available here), containing the solutions to all of the list function problems of week 1. Here is the protocol:
- Form a small group (2 to 4 people).
- Come to the front of the class to pick up a sheet of index cards (two pages).
- Cut the sheets into individual cards.
- Start working on exercises. The cards will be useful in “Part 1: Observation” and “Part 2: Conjecture”.
Higher-order functions on lists and trees ⭐️
You may have noted that we asked to write very similar code many, many times in last week’s exercises. Let’s explore this observation in more detail.
Part 1: Observation ⭐️
Let’s look at five categories of functions, each illustrated by a pair of recursive functions from last week’s exercises. In your stack of index cards, find the cards for the following functions:
sumandproduct(we’ll call this pair “associative”)incrementandcapAtZero(we’ll call this pair “map”)collectEvenandremoveZeroes(we’ll call this pair “filter”)countEvenandmultiplyOdd(we’ll call this pair “foldRight”)minandlast(we’ll call this pair “reduceRight”)
Read the implementation of these functions carefully. For each pair, ask yourself: what do these two functions have in common? Which parts differ? Use underlines, colors, or boxes to highlight the differences (you can write on the cards).
Once you’re done, put each of these pairs down, leaving some space around each pair to add more cards.
Part 2: Conjecture ⭐️
Let’s see how well the patterns that you have found generalize.
-
You should have about 20 cards left. Try to classify each card into one of the five categories above, based on how well it fits the pattern that you identified earlier. If you feel that you need more categories, that’s OK. Oh, and beware: two of the cards don’t fit!
Hint
In particular, you may decide that
allEven/allPositiveOrZeroandanyOdd/anyNegativeshould be their own categories. That’s fine!Hint
reverseAppendandinitare the ugly ducklings.reverseAppendis an instance of a different pattern (a “left fold”) that we will study later. -
Find another group of students, and compare your results with them. Do you agree on everything? Did you create the same additional categories? If you disagree, is it because one category is a special case of another one?
Hint
If you look very carefully, you will see that all categories are special cases of “fold” and “reduce”.
Here is a possible classification:
- “
associative”:sum,product - “
map”:decrement,increment,multiplyBy2,capAtZero - “
filter”:collectEven,removeOdd,removeZeroes,collectMultiples,intersection,difference - “
foldRight”:length,countPositive,countEven,multiplyOdd,horner,takeWhilePositive,append - “
reduceRight”:min,subtract,last - “
forall”:allPositiveOrZero,allEven,isSubset - “
exists”:anyOdd,anyNegative,contains
Part 3: Experiment ⭐️
Time for higher-order functions! Take the categories above one by one, and for each of them, write one single function that is more general than all the examples belonging to that category. To help with this, consider starting with the following warm-up exercise:
Warm-up
Higher-order functions can be used to “abstract away” part of a function.
-
Consider the following two functions, which check whether the head of a list has a certain property, and return false if the list is empty:
def headIsEven(l: IntList): Boolean = !l.isEmpty && l.head % 2 == 0 def headIsPositive(l: IntList): Boolean = !l.isEmpty && l.head > 0higher-order-functions/src/main/scala/hofs/fun.scala
The bodies of these functions are very similar, so we can abstract away the common parts into a separate higher-order function (a function that takes a function):
def headHasProperty(p: Int => Boolean, l: IntList): Boolean = !l.isEmpty && p(l.head)higher-order-functions/src/main/scala/hofs/fun.scala
Now, use
headHasPropertyto refactor (i.e. rewrite in a more succinct way) the functionsheadIsEvenandheadIsPositive:Solution
def headIsEven1(l: IntList): Boolean = headHasProperty(i => i % 2 == 0, l) def headIsPositive1(l: IntList): Boolean = headHasProperty(i => i > 0, l)higher-order-functions/src/main/scala/hofs/fun.scala
-
Now take a look at these three functions:
def DoubleTriple(x: Int) = IntCons(x * 2, IntCons(x * 3, IntNil())) def DivideTrivide(x: Int) = IntCons(x / 2, IntCons(x / 3, IntNil())) def IncrementDeuxcrement(x: Int) = IntCons(x + 1, IntCons(x + 2, IntNil()))higher-order-functions/src/main/scala/hofs/fun.scala
What parts do they have in common? What parts are different?
Write a single function
ConstructTwothat abstracts away the common parts of the three functions above, and rewrite all three functions to use it:def ConstructTwo(f: Int => Int, g: Int => Int): Int => IntList = ??? val DoubleTriple2 = TODO val DivideTrivide2 = TODO val IncrementDeuxcrement2 = TODOhigher-order-functions/src/main/scala/hofs/fun.scala
def ConstructTwo(f: Int => Int, g: Int => Int): Int => IntList =
x => IntCons(f(x), IntCons(g(x), IntNil()))
val DoubleTriple2 = ConstructTwo(x => x * 2, x => x * 3)
val DivideTrivide2 = ConstructTwo(x => x / 2, x => x / 3)
val IncrementDeuxcrement2 = ConstructTwo(x => x + 1, x => x + 2)
higher-order-functions/src/main/scala/hofs/fun.scala
Refactoring recursive functions
Your task is now to generalize this process of abstraction to the groups of functions that you have identified.
The general functions that you come up with will need additional parameters, just like in the refactoring exercise. They should be such that you can rewrite all the functions that fit into the categories as special cases of the general functions, by passing a few parameters.
Here is a series of steps to follow:
- Pick a category.
- Highlight the parts that differ between functions of that category.
- Rewrite all the functions to isolate the parts that differ, representing the differences as
vals. - Transform the new
vals into parameters of the function.
The following hint shows you this process step by step for “associative” (the category that contains sum and product), but don’t look just yet! Try it on your own and with other students first.
Hint: Step-by-step example
-
Let’s pick the “associative” category.
-
There are two differences:
- The base case (0 vs 1)
- The recursive case (+ vs *)
-
We isolate the + and * into
valfunctions, and the 0 and 1 into simplevals:l.head + sum(list.tail)becomesf(l.head, sum(list.tail)), wherefis(x, y) => x + y.l.head * product(list.tail)becomesf(l.head, product(list.tail))wherefis((x, y) => x * y).
The result is:
def sumRewritten(l: IntList): Int = val base = 0 val f = (x: Int, y: Int) => x + y if l.isEmpty then base else f(l.head, sumRewritten(l.tail)) def productRewritten(l: IntList): Int = val base = 1 val f = (x: Int, y: Int) => x * y if l.isEmpty then base else f(l.head, productRewritten(l.tail))higher-order-functions/src/main/scala/hofs/listOps.scala
-
Finally, we extract
baseandfinto parameters:def sp(l: IntList, base: Int, f: (Int, Int) => Int): Int = if l.isEmpty then base else f(l.head, sp(l.tail, base, f)) def sumDef(l: IntList): Int = sp(l, 0, (x, y) => x + y) def productDef(l: IntList): Int = sp(l, 1, (x, y) => x * y)higher-order-functions/src/main/scala/hofs/listOps.scala
Alternatively, we could also curry the last argument of
spto make the definitions ofsumandproductmore succinct:def spCurried(base: Int, f: (Int, Int) => Int): IntList => Int = def sp(l: IntList): Int = if l.isEmpty then base else f(l.head, sp(l.tail)) sp // Same as (l: IntList) => sp(l) val sumVal = spCurried(0, (x, y) => x + y) val productVal = spCurried(1, (x, y) => x * y)higher-order-functions/src/main/scala/hofs/listOps.scala
Your turn! Once you get used to it, you will find that almost all of last week’s functions can be succinctly reimplemented.
You will find that sometimes you cannot fully abstract across a whole category, because of types: for example, you will be able to write countEven and multiplyOdd using a single function, and allPositiveOrZero and allEven using a single function, but unifying the two will not work, due to mismatched types. It’s possible to unify them using a concept called polymorphism, which we will study in week 4.
Once you’re done with this exercise on paper, use the REPL, last week’s tests, or last week’s worksheet to confirm that your rewritten (“refactored”) functions work.
You can get the scaffold code by pulling from our course exercises repository with the following command:
cd ~/Documents/cs214/exercises # Replace with the exercises where you cloned the exercises
git pull
Note that it covers only the first part of the set (this part). For the second part (below), we have only provided you with a simple worksheet, with no tests, so that you can learn to test your functions directly yourself (either directly in the worksheet, or in the REPL).
def map(f: Int => Int)(l: IntList): IntList =
if l.isEmpty then IntNil()
else IntCons(f(l.head), map(f)(l.tail))
def filter(p: Int => Boolean)(l: IntList): IntList =
if l.isEmpty then IntNil()
else
val keep = p(l.head)
val tl = filter(p)(l.tail)
if keep then IntCons(l.head, tl) else tl
def foldRight(f: (Int, Int) => Int, base: Int)(l: IntList): Int =
if l.isEmpty then base
else f(l.head, foldRight(f, base)(l.tail))
def foldRightList(f: (Int, IntList) => IntList, base: IntList)(l: IntList): IntList =
if l.isEmpty then base
else f(l.head, foldRightList(f, base)(l.tail))
def reduceRight(f: (Int, Int) => Int)(l: IntList): Int =
if l.isEmpty then throw new IllegalArgumentException("Empty list!")
else if l.tail.isEmpty then l.head
else f(l.head, reduceRight(f)(l.tail))
higher-order-functions/src/main/scala/hofs/listOps.scala
def forall(p: Int => Boolean)(l: IntList): Boolean =
if l.isEmpty then true
else p(l.head) && forall(p)(l.tail)
def exists(p: Int => Boolean)(l: IntList): Boolean =
if l.isEmpty then false
else p(l.head) || exists(p)(l.tail)
higher-order-functions/src/main/scala/hofs/listOps.scala
val length = foldRight((_, acc) => 1 + acc, 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val allPositiveOrZero = forall(hd => hd >= 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val countPositive = foldRight(
(hd, acc) => (if hd > 0 then 1 else 0) + acc,
0
)
higher-order-functions/src/main/scala/hofs/listOps.scala
val sum = foldRight((hd, acc) => hd + acc, 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val product = foldRight((hd, acc) => hd * acc, 1)
higher-order-functions/src/main/scala/hofs/listOps.scala
val anyOdd = exists(hd => hd % 2 != 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val decrement = map(hd => hd - 1)
higher-order-functions/src/main/scala/hofs/listOps.scala
val collectEven = filter(hd => hd % 2 == 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val min = reduceRight(scala.math.min)
higher-order-functions/src/main/scala/hofs/listOps.scala
val increment = map(hd => hd + 1)
higher-order-functions/src/main/scala/hofs/listOps.scala
val subtract = reduceRight((hd, acc) => hd - acc)
higher-order-functions/src/main/scala/hofs/listOps.scala
val removeOdd = filter(hd => hd % 2 == 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val countEven = foldRight(
(hd, acc) => (if hd % 2 == 0 then 1 else 0) + acc,
0
)
higher-order-functions/src/main/scala/hofs/listOps.scala
val multiplyBy2 = map(hd => 2 * hd)
higher-order-functions/src/main/scala/hofs/listOps.scala
val anyNegative = exists(hd => hd < 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val allEven = forall(hd => hd % 2 == 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val multiplyOdd = foldRight(
(hd, acc) => (if hd % 2 != 0 then hd else 1) * acc,
1
)
higher-order-functions/src/main/scala/hofs/listOps.scala
def horner(x: Int, l: IntList) =
foldRight((hd, acc) => hd + x * acc, 0)(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
val capAtZero = map(hd => if hd > 0 then 0 else hd)
higher-order-functions/src/main/scala/hofs/listOps.scala
val removeZeroes = filter(hd => hd != 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
def reverseAppend(l1: IntList, l2: IntList): IntList =
if l1.isEmpty then l2
else reverseAppend(l1.tail, IntCons(l1.head, l2))
higher-order-functions/src/main/scala/hofs/listOps.scala
val takeWhilePositive =
foldRightList((hd, acc) => if hd > 0 then IntCons(hd, acc) else IntNil(), IntNil())
higher-order-functions/src/main/scala/hofs/listOps.scala
def append(l1: IntList, l2: IntList): IntList =
foldRightList(IntCons.apply, l2)(l1)
higher-order-functions/src/main/scala/hofs/listOps.scala
def collectMultiples(d: Int, l: IntList): IntList =
filter(hd => hd % d == 0)(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
val last = reduceRight((hd, acc) => acc)
higher-order-functions/src/main/scala/hofs/listOps.scala
def init(l: IntList): IntList =
if l.isEmpty then
throw IllegalArgumentException("Empty list!")
else if l.tail.isEmpty then IntNil()
else IntCons(l.head, init(l.tail))
higher-order-functions/src/main/scala/hofs/listOps.scala
def contains(l: IntList, n: Int): Boolean =
exists(hd => hd == n)(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
def isSubset(l: IntList, L: IntList): Boolean =
forall(hd => contains(L, hd))(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
def intersection(l: IntList, L: IntList): IntList =
filter(hd => contains(L, hd))(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
def difference(l: IntList, L: IntList): IntList =
filter(hd => !contains(L, hd))(l)
higher-order-functions/src/main/scala/hofs/listOps.scala
Part 4: Conclusion
The higher-order functions that we have discovered today are useful beyond Scala, and have names common across programming languages:
- “
associative” (sum/product), once suitably generalized, is almost identical to “foldRight”, so it typically doesn’t have its own name. - “
map” (increment/multiply) and “filter” (collectEven/removeOdd) are special cases offoldRight. They are so useful that they are typically defined (and optimized) separately fromfoldRight, with their own name. They are present in the vast majority of modern programming languages (including JavaScript, Python, and Java). - “
foldRight” (countEven/multiplyOdd) and “reduceRight” (minMax/last) are similar to each other, butreduceRightuses the last element of the list as its starting point, and does not work for empty lists.
Variants of these functions are found in many fields: in data science with MapReduce; in databases with SELECT/WHERE (filter); in graphics with GPUs and SIMD programs; and in many other computational sciences, where they help process large data sets. There are other such higher-order functions that capture common patterns, which we will explore throughout the course.
The supporting code for this week’s lab includes definitions of all of these functions, should you want to use them. We found them useful in our own solution.
-
🔜
mapandfilterare special cases offoldRight. Can you rewrite them usingfoldRightinstead of direct recursion?def mapAsFoldRight(f: Int => Int): IntList => IntList = ??? def filterAsFoldRight(p: Int => Boolean): IntList => IntList = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
⭐️
allEvenandanyNegativecan be implemented usingfoldRight, but is the result as efficient as the original function? Why or why not?Hint
Try evaluating each implementation with the substitution model and counting evaluation steps.
-
Another way to implement
allEvenandanyNegativewould be to define two functions:- one function,
forall, to check whether a predicate (a function fromInttoBoolean) holds (i.e., evaluates totrue) for all values in a list, and - one function,
exists, to check whether a predicate holds for any value in a list:
def forall(p: Int => Boolean)(l: IntList): Boolean = if l.isEmpty then true else p(l.head) && forall(p)(l.tail) def exists(p: Int => Boolean)(l: IntList): Boolean = if l.isEmpty then false else p(l.head) || exists(p)(l.tail)higher-order-functions/src/main/scala/hofs/listOps.scala
-
Rewrite
allEvenandanyNegativeusingforall/exists.def allEven(l: IntList): Boolean = if l.isEmpty then true else (l.head % 2 == 0) && allEven(l.tail)higher-order-functions/src/main/scala/hofs/listOps.scala
def anyNegative(l: IntList): Boolean = if l.isEmpty then false else l.head < 0 || anyNegative(l.tail)higher-order-functions/src/main/scala/hofs/listOps.scala
-
The two implementations provided above use
ifwith a constant branch (if … then true else …andif … then false else …). Can you simplify them to eliminate theifs?def forallNoIf(p: Int => Boolean)(l: IntList): Boolean = ??? def existsNoIf(p: Int => Boolean)(l: IntList): Boolean = ???higher-order-functions/src/main/scala/hofs/fun.scala
- one function,
def mapAsFoldRight(f: Int => Int): IntList => IntList =
foldRightList((hd, tl) => IntCons(f(hd), tl), IntNil())
def filterAsFoldRight(p: Int => Boolean): IntList => IntList =
foldRightList(
(hd, tl) =>
if p(hd) then IntCons(hd, tl) else tl,
IntNil()
)
higher-order-functions/src/main/scala/hofs/fun.scala
-
Less efficient, because the
foldRight-based implementation is not short-circuiting. This inefficiency is why many languages provide specialized alternativesfoldRightfor boolean predicates, variously calledforall/exists(Scala),all/any(Python),allMatch/anyMatch(Java), etc.
val allEven = forall(hd => hd % 2 == 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
val anyNegative = exists(hd => hd < 0)
higher-order-functions/src/main/scala/hofs/listOps.scala
def forallNoIf(p: Int => Boolean)(l: IntList): Boolean =
l.isEmpty || (p(l.head) && forallNoIf(p)(l.tail))
def existsNoIf(p: Int => Boolean)(l: IntList): Boolean =
!l.isEmpty && (p(l.head) || existsNoIf(p)(l.tail))
higher-order-functions/src/main/scala/hofs/fun.scala
Part 5: Going further
The functions that we have seen today are applicable beyond lists; we will look at this in more depth next week, but you may be interested to get started right away:
-
🔜 Look over the tree functions of week 1. Do you see patterns and categories similar to the ones we saw for lists? Could you operate the same generalization by defining map, filter, and reduce on trees?
-
🔜 Look over the string functions of week 1. Are they different from the list functions? Is there anything preventing you to use your list functions on strings?
Functions as values
Operations on functions ⭐️
-
Think about the types that you already know in Scala:
Boolean,Int,String, … (what else?): each of them has operations that you can use to combine them:*onInts,||onBooleans,+onStrings,==on most types… (what else?)Which ones of these operations make sense for functions? Do they make sense of all functions, or just for some functions? What could it mean to “or” or “add” two functions together?
Hint
Consider concrete examples: if I have two functions
isOddandisGreaterThan5, how can I combine them? What operators can I apply to their results?How about two functions $f: x ↦ x + 1$ and $g: x ↦ x^2$? The answer should be a bit different from the
Booleancase. Why? Consider the argument and return types.Hint
Take time to think about what
==may mean for functions. If we define equality asf(x) ≣ g(x)for allx, then could you write a program that checks whether two functions are equal? How long would that program run for? -
Can you think of operations that make sense for functions, but not for the other types you know?
Hint
Think again about $f: x ↦ x + 1$ and $g: x ↦ x^2$. I can add or multiply their results, of course, but what else can I do with them?
Then think about logical negation (
not),isEven, andisOdd. Can I define one in terms of the other two?
-
-
More types:
Float,Double,IntList,IntTree,cs214.Entry, … -
More operations: Unary negation (
-x) onInt,lengthon a String,reverseAppendon two lists, …
Functions can be lifted by combining their results with operators that apply to their return types. Two functions that return
Booleans can be combined into a single function that returns the||of the results of both functions; two functions that returnInts can be combined into a function that returns the sum of thoseInts, etc.: See the exercises below.==could be lifted to create a new function, so that(f ≡ g)(x)would be(f(x) == g(x)). For real equality (a single boolean that says whether both functions always return the same results), it depends; see the exercise titled “Equality” below. -
-
A single function can be applied: $(f, x) ↦ f(x)$ is a perfectly reasonable function.
Two functions can be composed by applying one, then the other: $g ∘ f: x ↦ g(f(x))$
Function combinations
Composition ⭐️
-
⭐️ Write a function that takes two functions
fandg, and returns a new function that applies them in sequence (f, theng). This is called composition and is usually writteng ∘ fin mathematics. In Scala, it is typically writteng `compose` forf `andThen` g(both are built-in):def andThen(f: Int => Double, g: Double => String): Int => String = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
Can you write variants of
composefor other types of functions?Bool => Bool, orString => String, for example. Does the implementation look different? -
More generally, assume that
fhas typeA => Bandghas typeC => D. Under what conditions can you composefandgto formg ∘ f? What aboutf ∘ g?
- Using an anonymous function:
def andThen(f: Int => Double, g: Double => String): Int => String =
(x: Int) => g(f(x))
higher-order-functions/src/main/scala/hofs/fun.scala
-
The implementation is the same; only the type signature changes. We will see in a future lecture how to eliminate this redundancy.
-
g ∘ fis valid if the return type offmatches the input type ofg(we will see later that the types don’t have to be strictly equal: subtyping, which we will study soon, is enough).
Identity
0 is the neutral element for +: $0 + x = x + 0 = x$ for all $x$. 1 is the neutral element for *.
What is the neutral element for compose? That is, which function id is such that id ∘ f ≡ f ∘ id ≡ f for all f? (we use f ≡ g here to mean that f(x) == g(x) for all x.
val id: Int => Int =
TODO
higher-order-functions/src/main/scala/hofs/fun.scala
val id: Int => Int =
x => x
higher-order-functions/src/main/scala/hofs/fun.scala
Flip
-
Define a function
flip. It takes a function and returns the same function, but with the arguments flipped.def flip(f: (Int, Int) => Int): (Int, Int) => Int = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
What happens if you compose
flipwith itself (in other words, what doesflip ∘ flipdo?)
def flip(f: (Int, Int) => Int): (Int, Int) => Int =
(x, y) => f(y, x)
higher-order-functions/src/main/scala/hofs/fun.scala
Modularizing with compositions ⭐
Higher-order functions are often very useful to capture repeated patterns. For example, here are some closely related functions:
val squareMinusOne = (x: Int) => (x - 1) * (x - 1)
val squarePlusOne = (x: Int) => (x + 1) * (x + 1)
val squareSquare = (x: Int) => (x * x) * (x * x)
val squareMinusTwo = (x: Int) => (x - 2) * (x - 2)
val squareSquareSquare = (x: Int) =>
((x * x) * (x * x)) * ((x * x) * (x * x))
higher-order-functions/src/main/scala/hofs/fun.scala
Can you define these functions in terms of the following ones?
val square = (x: Int) => x * x
val plusOne = (x: Int) => x + 1
val minusOne = (x: Int) => x - 1
def composeInt(f: Int => Int, g: Int => Int): Int => Int =
x => f(g(x))
higher-order-functions/src/main/scala/hofs/fun.scala
val squareMinusOne = composeInt(square, minusOne)
val squarePlusOne = composeInt(square, plusOne)
val squareSquare = composeInt(square, square)
val squareMinusTwo = composeInt(square, composeInt(minusOne, minusOne))
val squareSquareSquare = composeInt(square, composeInt(square, square))
higher-order-functions/src/main/scala/hofs/fun.scala
Lifting
-
Write a function that takes two functions
Int => Intand returns a new functionInt => Intwhose results are the sum of the results of the first two functions.def adder(f: Int => Double, g: Int => Double): Int => Double = ???higher-order-functions/src/main/scala/hofs/fun.scala
Hint
The result should be such that
adder(f, g)(x) == f(x) + g(x).This is called a lifted version of
+. In mathematics, this is the usual definition of+on functions. -
Can you do the same for
*,-,/, other operators? Do you notice any similarities?def multiplier(f: Int => Double, g: Int => Double): Int => Double = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
We saw previously that
0is the neutral element of+. What is the neutral element ofadder? In other words, what function is suchadder(f, g) ≡ f?
def adder(f: Int => Double, g: Int => Double): Int => Double =
x => f(x) + g(x)
higher-order-functions/src/main/scala/hofs/fun.scala
def multiplier(f: Int => Double, g: Int => Double): Int => Double =
x => f(x) * g(x)
higher-order-functions/src/main/scala/hofs/fun.scala
The neutral element for adder is (x: Int) => 0.0, and for multiplier it’s (x: Int) => 1.0
Heavy lifting
Let’s extract the common code between the functions in the previous exercise.
-
Write a function that takes a single function
op(a binary operator such as+) and returns a lifted version of that operation (likeadderabove).def lifter(op: (Double, Double) => Double): (Int => Double, Int => Double) => (Int => Double) = ???higher-order-functions/src/main/scala/hofs/fun.scala
lifter((x, y) => x + y)should be the same asadder. Look for the common parts in the implementation, and extract the ones that vary between them. -
Rewrite
adderand other related functions in terms of this one.val adder2 = TODO val multiplier2 = TODOhigher-order-functions/src/main/scala/hofs/fun.scala
def lifter(op: (Double, Double) => Double): (Int => Double, Int => Double) => (Int => Double) =
(f, g) => (x => op(f(x), g(x)))
higher-order-functions/src/main/scala/hofs/fun.scala
val adder2 = lifter((x, y) => x + y)
val multiplier2 = lifter((x, y) => x * y)
higher-order-functions/src/main/scala/hofs/fun.scala
Multi-lifting
-
Write a lifted version of the boolean
&&(and) operator.def meet(f: Int => Boolean, g: Int => Boolean): (Int => Boolean) = ???higher-order-functions/src/main/scala/hofs/fun.scala
Hint
This function should be such that
meet(f, g)(x) == f(x) && g(x).Functions that return booleans are often called “predicates” (in other words,
f: Int => Booleanis a predicate, andmeetabove combines two predicates into one). -
🔜 Generalize
meetto accept more than one predicate. That is, write a functionMeetwhich, given a list of predicates, returns a single predicate that lifts&&across all of the predicates.def Meet(l: IntPredicateList): (Int => Boolean) = ???higher-order-functions/src/main/scala/hofs/fun.scala
def meet(f: Int => Boolean, g: Int => Boolean): (Int => Boolean) =
x => f(x) && g(x)
higher-order-functions/src/main/scala/hofs/fun.scala
def Meet(l: IntPredicateList): (Int => Boolean) =
if l.isEmpty then x => true
else meet(l.head, Meet(l.tail))
higher-order-functions/src/main/scala/hofs/fun.scala
Values as functions: defs, vals and currying
The idea of currying is to change (slightly) the way a function is invoked (called) to make it easier to use. Let us explore this on two examples:
Translating between defs and vals (named and anonymous functions) ⭐️
Using the template below, rewrite (and show how to call) each of the following functions:
def isGreaterThanBasic(x: Int, y: Int): Boolean =
x > y
val isGreaterThanAnon: (Int, Int) => Boolean =
(x, y) => x > y
val isGreaterThanCurried: Int => Int => Boolean =
x => y => x > y // Same as `x => (y => x > y)`
def isGreaterThanCurriedDef(x: Int)(y: Int): Boolean =
x > y
// How to call:
// For all x, y:
// isGreaterThan(x, y)
// == isGreaterThanAnon(x, y)
// == isGreaterThanCurried(x)(y)
// == isGreaterThanCurriedDef(x)(y)
higher-order-functions/src/main/scala/hofs/fun.scala
-
incrHeadByXdef incrHeadByXBasic(x: Int, l: IntList): IntList = if l.isEmpty then l else IntCons(l.head + x, l.tail) val incrHeadByXAnon: (Int, IntList) => IntList = TODO val incrHeadByXCurried: Int => IntList => IntList = TODO def incrHeadByXCurriedDef(x: Int)(l: IntList): IntList = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
addToFrontdef addToFrontBasic(x: Int, y: Int, l: IntList): IntList = IntCons(x, IntCons(y, l)) val addToFrontAnon: (Int, Int, IntList) => IntList = TODO val addToFrontPartlyCurried: (Int, Int) => IntList => IntList = TODO val addToFrontCurried: Int => Int => IntList => IntList = TODO def addToFrontCurriedDef(x: Int)(y: Int)(l: IntList): IntList = ???higher-order-functions/src/main/scala/hofs/fun.scala
-
containsdef containsBasic(l: IntList, n: Int): Boolean = !l.isEmpty && (n == l.head || contains(l.tail, n)) def containsAnon: (IntList, Int) => Boolean = TODO def containsCurried: IntList => Int => Boolean = TODO def containsCurriedDef(l: IntList)(n: Int): Boolean = ???higher-order-functions/src/main/scala/hofs/fun.scala
It is debatable whether
containsAnonis really anonymous, since it must use its own name (the one being defined in thedef) for the recursive call. This recursion issue is also why we usedefinstead ofvalhere. -
headHasProperty(from “Part 3”, above)def headHasPropertyBasic(p: Int => Boolean, l: IntList): Boolean = !l.isEmpty && p(l.head) val headHasPropertyAnon: ((Int => Boolean), IntList) => Boolean = TODO val headHasPropertyCurried: (Int => Boolean) => IntList => Boolean = TODO def headHasPropertyCurriedDef(p: Int => Boolean)(l: IntList): Boolean = ???higher-order-functions/src/main/scala/hofs/fun.scala
Finally, using this new headHasPropertyCurried, rewrite headIsEven and headIsPositive. Which version is shorter?
def headIsEven(l: IntList): Boolean =
!l.isEmpty && l.head % 2 == 0
def headIsPositive(l: IntList): Boolean =
!l.isEmpty && l.head > 0
higher-order-functions/src/main/scala/hofs/fun.scala
val headIsEven2 =
TODO
val headIsPositive2 =
TODO
higher-order-functions/src/main/scala/hofs/fun.scala
def incrHeadByXBasic(x: Int, l: IntList): IntList =
if l.isEmpty then l
else IntCons(l.head + x, l.tail)
val incrHeadByXAnon: (Int, IntList) => IntList =
(x, l) =>
if l.isEmpty then l
else IntCons(l.head + x, l.tail)
val incrHeadByXCurried: Int => IntList => IntList =
x =>
l =>
if l.isEmpty then l
else IntCons(l.head + x, l.tail)
def incrHeadByXCurriedDef(x: Int)(l: IntList): IntList =
if l.isEmpty then l
else IntCons(l.head + x, l.tail)
higher-order-functions/src/main/scala/hofs/fun.scala
def addToFrontBasic(x: Int, y: Int, l: IntList): IntList =
IntCons(x, IntCons(y, l))
val addToFrontAnon: (Int, Int, IntList) => IntList =
(x, y, l) => IntCons(x, IntCons(y, l))
val addToFrontPartlyCurried: (Int, Int) => IntList => IntList =
(x, y) => l => IntCons(x, IntCons(y, l))
val addToFrontCurried: Int => Int => IntList => IntList =
x => y => l => IntCons(x, IntCons(y, l))
def addToFrontCurriedDef(x: Int)(y: Int)(l: IntList): IntList =
IntCons(x, IntCons(y, l))
higher-order-functions/src/main/scala/hofs/fun.scala
def containsBasic(l: IntList, n: Int): Boolean =
!l.isEmpty && (n == l.head || contains(l.tail, n))
def containsAnon: (IntList, Int) => Boolean =
(l, n) =>
!l.isEmpty && (n == l.head || containsAnon(l.tail, n))
def containsCurried: IntList => Int => Boolean =
l =>
n =>
!l.isEmpty && (n == l.head || containsCurried(l.tail)(n))
def containsCurriedDef(l: IntList)(n: Int): Boolean =
!l.isEmpty && (n == l.head || containsCurried(l.tail)(n))
higher-order-functions/src/main/scala/hofs/fun.scala
def headHasPropertyBasic(p: Int => Boolean, l: IntList): Boolean =
!l.isEmpty && p(l.head)
val headHasPropertyAnon: ((Int => Boolean), IntList) => Boolean =
(p, l) => !l.isEmpty && p(l.head)
val headHasPropertyCurried: (Int => Boolean) => IntList => Boolean =
p => l => !l.isEmpty && p(l.head)
def headHasPropertyCurriedDef(p: Int => Boolean)(l: IntList): Boolean =
!l.isEmpty && p(l.head)
higher-order-functions/src/main/scala/hofs/fun.scala
val headIsEven2 =
headHasPropertyCurried(i => i % 2 == 0)
val headIsPositive2 =
headHasPropertyCurried(i => i > 0)
higher-order-functions/src/main/scala/hofs/fun.scala
Note that in all these solutions the curried function can be written as a val with an anonymous function inside, but could equivalently be written as a def taking one argument and returning a function. In other words, the following three forms are very close:
val headHasPropertyCurried0: (Int => Boolean) => IntList => Boolean =
(p: Int => Boolean) => (l: IntList) => !l.isEmpty && p(l.head)
def headHasPropertyCurried1(p: Int => Boolean): IntList => Boolean =
(l: IntList) => !l.isEmpty && p(l.head)
def headHasPropertyCurried2(p: Int => Boolean)(l: IntList): Boolean =
!l.isEmpty && p(l.head)
higher-order-functions/src/main/scala/hofs/fun.scala
Currying container functions ⭐️
Now let’s see how these different styles can help:
Let’s assume we have a list of EPFL Sciper numbers registered for this course (students and staff), and a separate list of EPFL Scipers for just the staff. Thus, registered students are those who occur in the first list but not the second one:
val cs214All =
IntCons(123456, IntCons(654321, IntCons(111222, IntCons(333444, IntCons(555666, IntCons(787878, IntNil()))))))
val cs214Staff =
IntCons(654321, IntCons(333444, IntNil()))
higher-order-functions/src/main/scala/hofs/fun.scala
-
Write a function
isRegisteredForCS214that checks whether a given Sciper appears in thecs214Alllist. Write two versions: one usingcontainsBasic, as adef; and one usingcontainsCurried, as aval. For theval, do not create an anonymous function: the definition should have no mention of ascipervariable.def isRegisteredForCS214Def(sciper: Int): Boolean = ??? val isRegisteredForCS214Val = TODOhigher-order-functions/src/main/scala/hofs/fun.scala
-
Write a function
isCS214Studentthat checks whether a Sciper corresponds to a registered student (i.e. not a staff member). Write two versions: one usingcontainsBasic, as adef; and one usingcontainsCurried,notLifter, andandLifter, as aval. For theval, as before, do not create an anonymous function: the definition should have no mention of ascipervariable.def isCS214StudentDef(sciper: Int): Boolean = ???higher-order-functions/src/main/scala/hofs/fun.scala
def andLifter(f: Int => Boolean, g: Int => Boolean): Int => Boolean = n => f(n) && g(n) def notLifter(f: Int => Boolean): Int => Boolean = n => !f(n) val isCS214StudentVal = TODOhigher-order-functions/src/main/scala/hofs/fun.scala
The
valstyle is often called “point-free style”, which means using only function combinators likeandLifterandnotLifterinstead of explicit parameter names. -
Notice that the two versions of the function above will always scan both lists. Using the
differencefunction on lists, write a more general, curried function that takes two lists, computes the difference, and then returns a function that takes a sciper and validates it against the resulting list. Make sure that the list difference is computed once. Can you do it in point-free style, with no anonymous functions?def isCourseStudentDefPartlyCurried(all: IntList, staff: IntList): Int => Boolean = ???higher-order-functions/src/main/scala/hofs/fun.scala
def isRegisteredForCS214Def(sciper: Int): Boolean =
containsBasic(cs214All, sciper)
val isRegisteredForCS214Val =
containsCurried(cs214All)
higher-order-functions/src/main/scala/hofs/fun.scala
def isCS214StudentDef(sciper: Int): Boolean =
containsBasic(cs214All, sciper)
&& !containsBasic(cs214Staff, sciper)
higher-order-functions/src/main/scala/hofs/fun.scala
def andLifter(f: Int => Boolean, g: Int => Boolean): Int => Boolean =
n => f(n) && g(n)
def notLifter(f: Int => Boolean): Int => Boolean =
n => !f(n)
val isCS214StudentVal =
andLifter(
containsCurried(cs214All),
notLifter(containsCurried(cs214Staff))
)
higher-order-functions/src/main/scala/hofs/fun.scala
def isCourseStudentDefPartlyCurried(all: IntList, staff: IntList): Int => Boolean =
val students = difference(all, staff)
containsCurried(students)
higher-order-functions/src/main/scala/hofs/fun.scala
Equality 🔥
Mathematicians generally say that two functions are “equal” when they have the same outputs for all inputs.
-
By this definition, which of the following functions are equal?
val f0 = (x: Long) => x val f1 = (x: Long) => if x > 0 then x else -x val f2 = (x: Long) => x + 1 - 1 val f3 = (x: Long) => Math.sqrt(x.toDouble * x.toDouble).round val f4: Long => Long = x => if x < 0 then f4(x + 1) - 1 else if x > 0 then f4(x - 1) + 1 else 0higher-order-functions/src/main/scala/hofs/fun.scala
-
Can you define a function that checks whether two functions of type
Boolean => Booleanare equal?def eqBoolBool( f: Boolean => Boolean, g: Boolean => Boolean ) = ???higher-order-functions/src/main/scala/hofs/fun.scala
What about functions of the following types?
Int => BooleanBoolean => IntIntList => BooleanBoolean => IntListInt => Boolean => IntList
-
Can you come up with a general result? For which types
AandBcan one write a functioneqABthat checks whether two functions of typeA => Breturn the same outputs for all inputs? -
What do you think of this definition of equality? Is
f4really “the same” asf0, or do they differ in any way?
-
f0 ≡ f2 ≡ f4.f1 ≢ f3, because of rounding errors. -
Yes; it suffices to check whether
fandgagree ontrueandfalse:def eqBoolBool( f: Boolean => Boolean, g: Boolean => Boolean ) = f(true) == g(true) && f(false) == g(false)higher-order-functions/src/main/scala/hofs/fun.scala
Int => Boolean: yesBoolean => Int: yesIntList => Boolean: noBoolean => IntList: yesInt => Boolean => IntList: yes
(see below for why)
-
A sufficient condition is that
Ais finite andBsupports equality. -
f4is much slower thanf0.
Fixed points
-
A value
xis a fixed point offiff(x) == x. Do the following functions have fixed points? If so, which one(s)?val a = (x: Int) => x val b = (x: Int) => -x val c = (x: Int) => x + 1 val d = (x: Int) => (x / 2) + 5 val e = (x: Int) => if x % 10 == 0 then x else (x + 1) val f = (x: Int) => -(x * x) val g = (x: Int) => /* 🔥 */ /* assuming x > 0 */ if x == 1 then 1 else if x % 2 == 0 then x / 2 else 3 * x + 1higher-order-functions/src/main/scala/hofs/fun.scala
-
Write a function
fixedPointthat takes a functionfand an integerx, checks whetherxis already a fixpoint off, and then looks for a fixed point by repeatedly callingf, until it converges.def fixedPoint(f: Int => Int, start: Int): Int = ???higher-order-functions/src/main/scala/hofs/fun.scala
For example,
fixedPoint(((x: Int) => x / 2 + 5), 20)will call itself recursively withx = 15(20 / 2 + 5), thenx = 12(15 / 2 + 5), thenx = 11(12 / 2 + 5), thenx = 10(11 / 2 + 5). -
For each of the following expressions, indicate whether it terminates, and if so, what value is returned:
fixedPoint(((x: Int) => x / 2), 4)fixedPoint(((x: Int) => -x), 3)fixedPoint(((x: Int) => x), 123456)fixedPoint(((x: Int) => x + 1), 0)fixedPoint(((x: Int) => if (x % 10 == 0) then x else x + 1), 35)fixedPoint(((x: Int) => x / 2 + 5), 20)
What happens when there is no fixed point? Does
fixedPointwork for all functions above that have a fixed point? Does it depend on the starting value ofx? -
🔥 Finally, a question that is not directly relevant to the class but interesting to think about. For which functions and which inputs does
fixedPointwork?
-
- a. All integers are fixed points of this function
- b. 0 (and every other point is on a cycle)
- c. This function does not have fixed points
- d. 10… but also 9 (!) Why?
- e. All multiples of 10.
- f. 0 and -1
- g. 1
@tailrec
def fixedPoint(f: Int => Int, start: Int): Int =
val next = f(start)
if next == start then start else fixedPoint(f, next)
higher-order-functions/src/main/scala/hofs/fun.scala
-
- reduces to 0
- does not terminate
- reduces to 123456
- does not terminate
- reduces to 40
- reduces to 10
Convergence depends on the starting point (to illustrate this, try the last example starting from 5 instead of 20)
-
All functions that have fixed points that are also attractors, provided that the starting point is within the basin of attraction of an attracting fixed point.
It is currently unknown whether the basin of attraction of
1for the last function above (g. 🔥) covers all positive numbers (in other words, whetherfixedPointconverges to1for all positive inputs).
Debugging
The best way to improve your debugging skills is to apply the debugging guide to your own code in the labs. For more focused training, however, we provide a few exercises below. Happy hacking!
It may sound surprising for a debugging exercise set, but even here, we strongly encourage you to think about the exercises on paper first, in groups. Only then should you start working with the actual code.
Time for some recursion debugging
The following program is intended to compute and output $x^n$ by using the square-and-multiply method, but a bug has sneaked in. Can you find it, explain what went wrong, and fix it?
def squareAndMultiply(x: Double, n: Int): Double =
if n < 0 then
squareAndMultiply(1 / x, -n)
else if x == 1 then
1
else if n == 0 then
0
else if n % 2 == 0 then
squareAndMultiply(x * x, n / 2)
else
x * squareAndMultiply(x, n - 1)
debugging/src/main/scala/debugging/SquareAndMultiply.scala
By running sbt test or running the function manually with some input, we confirm there is a problem by observing wrong results.
For example, we would expect $2^{10}=1024$, but instead we get squareAndMultiply(2,10) == 0, which is mathematically wrong. We decide to simplify the input, one parameter at a time.
By simplifying x, we get 0 ** 10 and 1 ** 10, but both return the right result. So we simplify the exponent n instead: since the function is recursive on n, smaller values lead to fewer recursive calls. We try with 2 ** 0, which returns the wrong result.
Now that we have a simplified input that still gives wrong results, what we can do is run the function by hand on a piece of paper to see where it goes wrong. By doing this for the inputs (2,0), we get:
// 1. Start evaluating the expression
squareAndMultiply(2,0)
// 2. Take the right if-branch, where n == 0
0
// We're done
We see the function indeed evaluates to 0 with those arguments, but the result should be 1, mathematically speaking. Thus, the base case is wrong: when n (the power to which we want to raise x) is 0, the result should be 1.
Since the base case is wrong, all recursive calls that reduce down to that base case also end up outputting wrong results.
To fix the program, we replace 0 with 1 in the n == 0 base case.
def squareAndMultiply(x: Double, n: Int): Double =
if n < 0 then
squareAndMultiply(1 / x, -n)
else if x == 1 then
1
else if n == 0 then
1
else if n % 2 == 0 then
squareAndMultiply(x * x, n / 2)
else
x * squareAndMultiply(x, n - 1)
debugging/src/main/scala/debugging/SquareAndMultiply.scala
Encoding issues ⭐️
Prof. Pit-Claudel recently received a spam email with the following title:
Re: Design Query for √âcole polytechnique f√(c)d√(c)rale de ...
Explain what happened.
Let’s assume that the original title was:
Re: Design Query for École polytechnique fédérale de Lausanne
It appears that three things happened:
Éwas converted to√â- Both
éwere converted to√(c) - The message title was truncated to 60 characters, with the rest replaced by …
Which conversion issue could have caused É to become √â? We know from the first debugging lecture that this is most likely text encoded in one way and decoded in another way. We also know that we are looking for a multi-byte encoding and a single-byte decoding since the result has two characters. Let’s use the same trick as before to find a pair of such encodings:
import java.nio.charset.Charset
import scala.jdk.CollectionConverters.*
def findPairs(src: String, dst: String): Iterable[(Charset, Charset)] =
Charset.availableCharsets()
.values().asScala
.filter(enc => enc.canEncode())
.flatMap(enc =>
Charset.availableCharsets()
.values().asScala
.filter(dec => dec.decode(enc.encode(src)).toString() == dst)
.map(dec => (enc, dec))
)
debugging/src/main/scala/debugging/Encoding.scala
With these we can find pairs:
findPairs("É", "√â").foreach(println)
// (CESU-8,x-MacCroatian)
// (CESU-8,x-MacIceland)
// (CESU-8,x-MacRoman)
// (CESU-8,x-MacRomania)
// (CESU-8,x-MacTurkish)
// (UTF-8,x-MacCroatian)
// (UTF-8,x-MacIceland)
// (UTF-8,x-MacRoman)
// (UTF-8,x-MacRomania)
// (UTF-8,x-MacTurkish)
Based on this, we can guess that the incorrect decoding possibly happened on a macOS machine configured to use legacy encoding. We can further refine our experiment by checking whether the result is consistent for 'é':
findPairs("É", "√â").foreach((enc, dec) =>
println(f"$enc, $dec: é → ${dec.decode(enc.encode("é"))}")
)
// CESU-8, x-MacCroatian: é → √Š
// CESU-8, x-MacIceland: é → √©
// CESU-8, x-MacRoman: é → √©
// CESU-8, x-MacRomania: é → √©
// CESU-8, x-MacTurkish: é → √©
// UTF-8, x-MacCroatian: é → √Š
// UTF-8, x-MacIceland: é → √©
// UTF-8, x-MacRoman: é → √©
// UTF-8, x-MacRomania: é → √©
// UTF-8, x-MacTurkish: é → √©
This lets us eliminate two more pairs, and confirm the results of the other ones… except for one thing! é becomes √©, not √(c) on all the encodings. The most likely conclusion at this point is that the spammer’s system has a long sequence of broken email filters, one of which applies the wrong decoding algorithm (a legacy Mac encoding), and the other which replaced “special” characters such as © by “safe” equivalents.
🔜 We will see a more succinct syntax for this in a few weeks, called Comprehensions. The function findPairs can be rewritten as:
def findPairsSuccint(src: String, dst: String): Iterable[(Charset, Charset)] =
for
enc <- Charset.availableCharsets().values.asScala
if enc.canEncode()
dec <- Charset.availableCharsets().values.asScala
if dec.decode(enc.encode(src)).toString() == dst
yield (enc, dec)
debugging/src/main/scala/debugging/Encoding.scala
Debugging warnings
When Prof. Pit-Claudel demoed sbt new in class, the following appeared on screen:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
What happened?
“See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.”
This warning message is reported by slf4j-api version 1.7.x and earlier when the
org.slf4j.impl.StaticLoggerBinderclass could not be loaded into memory. This happens when no appropriate SLF4J binding could be found on the class path. Placing one (and only one) of slf4j-nop.jar[,] slf4j-simple.jar, slf4j-log4j12.jar, slf4j-jdk14.jar or logback-classic.jar on the class path should solve the problem.
You should be able to understand most of this explanation already. There are two technical terms that may be unfamiliar, which you can also look up:
-
class path: According to the Oracle documentation,
The class path is the path that the Java Runtime Environment (JRE) searches for classes and other resource files. […] The class search path (class path) can be set using either the
-classpathoption when calling a JDK tool (the preferred method) or by setting theCLASSPATHenvironment variable.(Reminder: by default, Scala uses the JRE.) The manual pages for the
scalacorjavaccommands have more information under the-classpathflag. -
binding: According to Wikipedia,
In programming and software design, a binding is an application programming interface (API) that provides glue code specifically made to allow a programming language to use a foreign library or operating system service (one that is not native to that language).
Debugging with the substitution model ⭐️
Lists
removeDuplicates
-
removeDuplicatestakes a list and produces a new list with all duplicated elements removed, keeping only the first one. For simplicity, we use1 :: 2 :: 3 :: IntNilto denoteIntCons(1, IntCons(2, IntCons(3, IntNil))). For example, for a list1 :: 2 :: 4 :: 2 :: 3 :: 3 :: IntNil, the second2and3are duplicated elements, so the returned list should be1 :: 2 :: 4 :: 3 :: IntNil.The following is an (incorrect) implementation of
removeDuplicates, using a functioncontains(l, x)that checks whether listlcontains valuex:def removeDuplicates(l: IntList): IntList = l match case IntNil() => IntNil() case IntCons(hd, tl) => if contains(tl, hd) then removeDuplicates(tl) else IntCons(hd, removeDuplicates(tl))debugging/src/main/scala/debugging/ListOps.scala
-
What is the expected return value of this program for the input
1 :: 2 :: 1 :: IntNil? -
Write down, step by step, the evaluation of
removeDuplicates(IntCons(1, IntCons(2, IntCons(1, IntNil())))), using the substitution model. You may assume thatcontainscorrectly computes its result and substitute calls to it in a single step.(Consider using an erasable surface, such as a whiteboard, tablet, or writing slate, to reduce repetition).
Where does the computation diverge from the expected result?
-
Fix the implementation.
-
-
1 :: 2 :: IntNil(only the first1is kept). -
Here is the result of evaluating the program step by step.
removeDuplicates(1 :: (2 :: (1 :: IntNil()))) ≡ if contains(2 :: (1 :: IntNil()), 1) then removeDuplicates(2 :: (1 :: IntNil())) else (1 :: removeDuplicates(2 :: (1 :: IntNil()))) ≡ if true then removeDuplicates(2 :: (1 :: IntNil())) else (1 :: removeDuplicates(2 :: (1 :: IntNil()))) ≡ removeDuplicates(2 :: (1 :: IntNil())) ≡ if contains(1 :: IntNil(), 2) then removeDuplicates(1 :: IntNil()) else (2 :: removeDuplicates(1 :: IntNil())) ≡ if false then removeDuplicates(1 :: IntNil()) else (2 :: removeDuplicates(1 :: IntNil())) ≡ 2 :: removeDuplicates(1 :: IntNil()) ≡ 2 :: (if contains(IntNil(), 1) then removeDuplicates(IntNil()) else (1 :: removeDuplicates(IntNil()))) ≡ 2 :: (if false then removeDuplicates(IntNil()) else (1 :: removeDuplicates(IntNil()))) ≡ 2 :: (1 :: removeDuplicates(IntNil())) ≡ 2 :: (1 :: IntNil())The expected output is
1 :: (2 :: IntNil()). Therefore, the implementation is not correct.Bug: In the
IntCons(hd, tl)case, whenhdis a duplicated element (i.e., it appears intlas well), the implementation removeshdinstead of removing all the duplicated ones intl. -
Here are two possible implementations:
def removeDuplicates(l: IntList): IntList = def loop(l: IntList, seen: IntList): IntList = l match case IntNil() => IntNil() case IntCons(hd, tl) => if contains(seen, hd) then loop(tl, seen) else IntCons(hd, loop(tl, IntCons(hd, seen))) loop(l, IntNil())debugging/src/main/scala/debugging/ListOps.scala
def remove(l: IntList, n: Int): IntList = l match case IntNil() => IntNil() case IntCons(`n`, tl) => remove(tl, n) case IntCons(m, tl) => IntCons(m, remove(tl, n)) def removeDuplicates_alt(l: IntList): IntList = l match case IntNil() => IntNil() case IntCons(hd, tl) => IntCons(hd, removeDuplicates_alt(remove(tl, hd)))debugging/src/main/scala/debugging/ListOps.scala
Binary search trees
isBST
A Binary Search Tree (BST) is a binary tree such that the value carried by each node is greater than all values in the left subtree and smaller than all values in the right subtree. For example, the following is a binary search tree:
8
/ \
3 10
/ \ \
1 6 14
-
Can a binary search tree contain duplicate values?
-
Is the following tree a BST?
val t0 = IntBranch(5, IntBranch(2, IntEmptyTree(), IntEmptyTree()), IntBranch(10, IntBranch(4, IntEmptyTree(), IntEmptyTree()), IntEmptyTree()))) -
The function
isBSTtakes a binary tree and returns true if the tree is a binary search tree, false otherwise. Here is one candidate implementation of it:def isBST(t: IntTree): Boolean = t match case IntEmptyTree() => true case IntBranch(v, l, r) => (l == IntEmptyTree() || v > l.value) && (r == IntEmptyTree() || v < r.value) && isBST(l) && isBST(r)debugging/src/main/scala/debugging/TreeOps.scala
-
There is something wrong with this implementation. But what? Using the substitution method, show what
isBST(t0)evaluates to (t0refers to the tree above):For succinctness, we recommend writing
IntEmptyTreeasε,IntBranch(v, l, r)as[v, l, r], andIntBranch(v, IntEmptyTree, IntEmptyTree)as[v], so that this input is written as follows:isBST([5 [2] [10 [4] ε]])
-
-
Fix the implementation.
-
No: “less than” and “greater than” are strict.
-
This tree is not a BST, because the
4is larger than5, yet is found on the right side of the5. -
Here are the first few steps; writing this on a tablet or whiteboard will be much more pleasant!
isBST([5 [2] [10 [4] ε]]) ≡ ([2] == ε || 5 > [2].value) && ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ (false || 5 > [2].value) && ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ 5 > [2].value && ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ 5 > 2 && ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ true && ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ ([10 [4] ε] == ε || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ (false || 5 < [10 [4] ε].value) && isBST([2]) && isBST([10 [4] ε]) ≡ 5 < [10 [4] ε].value && isBST([2]) && isBST([10 [4] ε]) ≡ 5 < 10 && isBST([2]) && isBST([10 [4] ε]) ≡ true && isBST([2]) && isBST([10 [4] ε]) ≡ isBST([2]) && isBST([10 [4] ε]) ≡ (ε == ε || 2 > ε.value) && (ε == ε || 2 < ε.value) && isBST(ε) && isBST(ε) && isBST([10 [4] ε]) ≡ (true || 2 > ε.value) && (ε == ε || 2 < ε.value) && isBST(ε) && isBST(ε) && isBST([10 [4] ε]) ≡ true && (ε == ε || 2 < ε.value) && isBST(ε) && isBST(ε) && isBST([10 [4] ε]) ≡ (ε == ε || 2 < ε.value) && isBST(ε) && isBST(ε) && isBST([10 [4] ε]) ≡ ... (similar steps) ≡ trueThe implementation returns
trueeven though the input tree is not a binary search tree.Bug: For each node
n, saynhas valuev, every value in the left subtreelshould be less thanv. It’s not enough to just check the root ofl, because the right subtree oflmay contain values bigger thanv. Same thing forn’s right subtree. -
Here is a correct implementation.
def forall(p: Int => Boolean)(t: IntTree): Boolean = t match case IntEmptyTree() => true case IntBranch(v, l, r) => p(v) && forall(p)(l) && forall(p)(r) def isBST(t: IntTree): Boolean = t match case IntEmptyTree() => true case IntBranch(v0, l, r) => forall(v1 => v0 > v1)(l) && isBST(l) && forall(v1 => v0 < v1)(r) && isBST(r)debugging/src/main/scala/debugging/TreeOps.scala
Bonus question: this implementation isn’t very efficient. Can you see why? How could you make this function better?
insert
Binary search trees are convenient to represent sets of values. To use them in this way, we need a contains function and an insert function.
Let’s focus on the insert function. insert takes a binary search tree representing a set of values $V$ and a new value $v_1$, and returns a new binary search tree representing the set $V \cup {v_1}$.
The following is an attempt at implementing insert:
def insert(t: IntTree, v1: Int): IntTree =
t match
case IntEmptyTree() => IntBranch(v1, IntEmptyTree(), IntEmptyTree())
case IntBranch(v0, l, r) =>
if v1 < v0 then IntBranch(v0, insert(l, v1), r)
else IntBranch(v0, l, insert(r, v1))
debugging/src/main/scala/debugging/TreeOps.scala
-
Find a counterexample: a binary search tree
tand an inputv1such thatinsert(t, v1)produces an incorrect result. -
Fix the implementation.
The issue with the original implementation is that it creates duplicate values, which violate the BST invariant. Here is a corrected implementation:
def insert(t: IntTree, v1: Int): IntTree =
t match
case IntEmptyTree() => IntBranch(v1, IntEmptyTree(), IntEmptyTree())
case IntBranch(v0, l, r) =>
if v0 > v1 then IntBranch(v0, insert(l, v1), r)
else if v0 < v1 then IntBranch(v0, l, insert(r, v1))
else t
debugging/src/main/scala/debugging/TreeOps.scala