Google Calendar API V3 android - Get all day event

I'm trying to get the Free busy data of other people that are within my Google organization using the google-api-services-calendar:v3 for Android (using Kotlin). I'm getting the times just fine for events with a set duration. But all day events don't show up on the list. Documentation on this is almost nowhere to be found and the stuff that I find on developers.google.com contains code that was deprecated in 2013...

// ...
val busyTimesList = mutableListOf<AgendaPlotter.TimeSpan>()
    SessionService.sharedInstance.getGoogleAccount(activity)
        observeOn(Schedulers.io())
        .subscribe {
            mCredential!!.selectedAccount = it.account
            val request = FreeBusyRequest()

            val durationCal = Calendar.getInstance()
            durationCal.time = startDay.time
            Calendars.startOfDay(durationCal)
            request.timeMin = DateTime(durationCal.time)
            durationCal.add(Calendar.DATE, 1)
            request.timeMax = DateTime(durationCal.time)


            val requestItems = listOf(FreeBusyRequestItem().setId("email@from.colleague"))
            request.items = requestItems
            request.timeZone = TimeZone.getDefault().id

            val busyTimes: FreeBusyResponse
               try {
                    val query = mService!!.freebusy().query(request)
                    // Use partial GET to retrieve only needed fields.
                    query.fields = "calendars"
                    busyTimes = query.execute()
                    busyTimes.calendars.forEach {
                    it.toPair().second.busy.forEach { timeSpan ->
                        val busyTime = AgendaPlotter.TimeSpan()
                        busyTime.fromTime.timeInMillis = timeSpan.start.value
                        busyTime.toTime.timeInMillis = timeSpan.end.value
                        busyTimesList.add(busyTime)
                    }
                 }
                 emitter.onNext(busyTimesList)
             } catch (e: IOException) {
                 e.printStackTrace()
                 // ...
             }
         }
// ...

So my question, how do I also obtain the whole day events?

After some searching I noticed that there is actually nothing wrong with the API. It's a setting for a whole day event to be busy of free.

By default this is set to free, which makes it not show as a busy time, which makes sense. This goes unnoticed by a lot of people and they will be "free" on that day.