Upgrade realm in an Android project

Question

I'm currently running Realm Version 0.82.0 in one of my Android projects. I didn't touch Realm for quite some time, until I recently noticed that they went up until version 2.0.2 in the meantime. I would like to upgrade my version of Realm, unfortunately, I don't know if the upgrade from my old version to the current release will be working or breaking my code.

I'm especially concerned of migrations, since the API for migrations seemed to have changed a bit since my code, and I'm unsure if my migrations will break if I just update my version. Unfortunately, there is no documentation about upgrading Realm version available on their webpage.

Does anyone have any experience with upgrading Realm, especiall a version increase over two major versions?

Answer 1

The main upgrade to Realm from 4+ to 5+ needs to change from:

realm.where(example.class)
    .findAllSorted("field")

To:

realm.where(example.class)
    .sort("field")
    .findAll();

Answer 2

The list of breaking changes is available in the CHANGELOG.MD on their Github.

However, it's worth noting that there were quite a few breaking changes on the road, especially noting 0.89.0.

From 0.82.0 to 5.1.0 is the following (which is the most stable version at the moment):

0.82.0:

BREAKING CHANGE: Fields with annotation @PrimaryKey are indexed automatically now. Older schemas require a migration.

(0.82.2 was most stable here, but it didn't work on Blackberry devices. The first stable version to use on Blackberry was 0.87.2.)

In 0.86.0+, you can add an index to the annotated field using

@Override
public void migrate(final DynamicRealm realm, long oldVersion, long newVersion) {
    RealmSchema schema = realm.getSchema();
    // version check and stuff
    RealmObjectSchema personSchema = schema.get("Person");
    personSchema.addIndex("fieldName");

0.83:

BREAKING CHANGE: Database file format update. The Realm file created by this version cannot be used by previous versions of Realm.

BREAKING CHANGE: Removed deprecated methods and constructors from the Realm class.

BREAKING CHANGE: Introduced boxed types Boolean, Byte, Short, Integer, Long, Float and Double. Added null support. Introduced annotation @Required to indicate a field is not nullable. String, Date and byte[] became nullable by default which means a RealmMigrationNeededException will be thrown if an previous version of a Realm file is opened.

Oh boy, this is a nice one. NULL support.

Boxed types for primitives became available. Boxed types are nullable by default. All String, Date, and byte[] must be annotated with @Required, or schema.setNullable("fieldName", nullability) and make them all nullable.

0.84.0:

Async queries were added. Nothing new here in terms of schema.

0.85.0:

BREAKING CHANGE: Removed RealmEncryptionNotSupportedException since the encryption implementation changed in Realm's underlying storage engine. Encryption is now supported on all devices.

BREAKING CHANGE: Realm.executeTransaction() now directly throws any RuntimeException instead of wrapping it in a RealmException (#1682).

BREAKING CHANGE: RealmQuery.isNull() and RealmQuery.isNotNull() now throw IllegalArgumentException instead of RealmError if the fieldname is a linked field and the last element is a link (#1693).

Nothing important here yet, although:

Setters in managed object for RealmObject and RealmList now throw IllegalArgumentException if the value contains an invalid (unmanaged, removed, closed, from different Realm) object (#1749).

This one is an interesting one. Previously it just failed, so this is for the best. But it is Realm's largest limitation, too.

0.86.0:

BREAKING CHANGE: The Migration API has been replaced with a new API.

BREAKING CHANGE: RealmResults.SORT_ORDER_ASCENDING and RealmResults.SORT_ORDER_DESCENDING constants have been replaced by Sort.ASCENDING and Sort.DESCENDING enums.

BREAKING CHANGE: RealmQuery.CASE_SENSITIVE and RealmQuery.CASE_INSENSITIVE constants have been replaced by Case.SENSITIVE and Case.INSENSITIVE enums.

BREAKING CHANGE: Realm.addChangeListener, RealmObject.addChangeListener and RealmResults.addChangeListener hold a strong reference to the listener, you should unregister the listener to avoid memory leaks.

BREAKING CHANGE: Removed deprecated methods RealmQuery.minimum{Int,Float,Double}, RealmQuery.maximum{Int,Float,Double}, RealmQuery.sum{Int,Float,Double} and RealmQuery.average{Int,Float,Double}. Use RealmQuery.min(), RealmQuery.max(), RealmQuery.sum() and RealmQuery.average() instead.

BREAKING CHANGE: Removed RealmConfiguration.getSchemaMediator() which is public by mistake. And RealmConfiguration.getRealmObjectClasses() is added as an alternative in order to obtain the set of model classes (#1797).

BREAKING CHANGE: Realm.addChangeListener, RealmObject.addChangeListener and RealmResults.addChangeListener will throw an IllegalStateException when invoked on a non-Looper thread. This is to prevent registering listeners that will not be invoked.

Added new Dynamic API using DynamicRealm and DynamicRealmObject.

Added Realm.getSchema() and DynamicRealm.getSchema().

The new Migration API, using DynamicRealm instead of Realm.getTable().

Some stuff were renamed, and you ought to unregister your change listeners if your result set is still valid. But it's worth noting that you should still retain a field variable to your RealmResults, because Realm's Context only has a weak reference to it.

0.87.0:

RX support. Nothing important.

0.87.2:

Removed explicit GC call when committing a transaction (#1925).

Finally, Realm got stable again! :)

0.88.0:

Breaking changes

Realm has now to be installed as a Gradle plugin.

DynamicRealm.executeTransaction() now directly throws any RuntimeException instead of wrapping it in a RealmException (#1682).

DynamicRealm.executeTransaction() now throws IllegalArgumentException instead of silently accepting a null Transaction object.

String setters now throw IllegalArgumentException instead of RealmError for invalid surrogates.

DynamicRealm.distinct()/distinctAsync() and Realm.distinct()/distinctAsync() now throw IllegalArgumentException instead of UnsupportedOperationException for invalid type or unindexed field.

All thread local change listeners are now delayed until the next Looper event instead of being triggered when committing.

Removed RealmConfiguration.getSchemaMediator() from public API which was deprecated in 0.86.0. Please use RealmConfiguration.getRealmObjectClasses() to obtain the set of model classes (#1797).

Realm.migrateRealm() throws a FileNotFoundException if the Realm file doesn't exist.

It is now required to unsubscribe from all Realm RxJava observables in order to fully close the Realm (#2357).

Welp. It's an AAR now. You have to add to classpath and run it with apply plugin: 'realm-android' instead of compile ... dependency.

Change listeners are only called on the next event loop, instead of immediately after commit. I'm.. honestly not entirely sure of the ramifications of this, but it means change listeners don't work on background threads. Only on looper threads (primarily the UI thread).

Enhancements

Support for custom methods, custom logic in accessors, custom accessor names, interface implementation and public fields in Realm objects (#909).

Improved .so loading by using ReLinker.

This is quite necessary though, so I wouldn't want to get stuck on 0.87.5 for sure.

0.89.0:

Breaking changes

@PrimaryKey field value can now be null for String, Byte, Short, Integer, and Long types. Older Realms should be migrated, using RealmObjectSchema.setNullable(), or by adding the @Required annotation. (#2515).

RealmResults.clear() now throws UnsupportedOperationException. Use RealmResults.deleteAllFromRealm() instead.

RealmResults.remove(int) now throws UnsupportedOperationException. Use RealmResults.deleteFromRealm(int) instead.

RealmResults.sort() and RealmList.sort() now return the sorted result instead of sorting in-place.

RealmList.first() and RealmList.last() now throw ArrayIndexOutOfBoundsException if RealmList is empty.

Removed deprecated method Realm.getTable() from public API.

Realm.refresh() and DynamicRealm.refresh() on a Looper no longer have any effect. RealmObject and RealmResults are always updated on the next event loop.

Okay, this one is the most messy one.

1.) you must add @Required annotation for @PrimaryKey annotated fields, because null is a valid primary key value.

2.) realm.refresh() no longer works. It will be removed anyways. Here's a workaround for 1.1.1 though, should be used only on background threads. It is available in Realm 3.2 again, though.

3.) getTable() is removed. Don't use it. Use the new migration API.

4.) realmResults.sort() returns a new RealmResults, which needs to have the change listener appended to it as well. I think it's unreliable, so I'd just use findAllSorted() instead.

5.) You might not think much of it, but

RealmObject and RealmResults are always updated on the next event loop. (NOTE: THIS IS NO LONGER TRUE SINCE REALM 3.1+ WHERE CHANGE LISTENERS ARE CALLED AGAIN ON commitTransaction())

This literally meant that RealmResults were only updated when the event loop occured, it was NOT immediately updated when you call realm.commitTransaction(). This also means that on background threads, the RealmResults did NOT update when you commitTransaction(), you had to requery them.

The RealmResults are only known to be updated after the appended RealmChangeListener is called. In 1.1.1, when the RealmChangeListener is called, all Results had been updated.

This change however also changed iteration behavior in transactions. In transactions, you always saw the newest version. This meant that a query was re-evaluated as you were iterating on it, and modifying elements. (THIS IS ALSO THE CASE SINCE REALM 3.0)

Example, previously this was valid code:

RealmResults<Stuff> stuffs = realm.where(Stuff.class).equalTo("something", false).findAll();
while(!stuffs.isEmpty()) {
    stuffs.get(0).setSomething(true);
}
// you end up here because stuffs will be empty 
// because of live auto-updates in transactions

However, this will no longer work. For me, this caused issues because I iterated sometimes like this

RealmResults<Stuff> stuffs = realm.where(Stuff.class).equalTo("something", false).findAll();
for(int i = 0; i < stuffs.size(); i++) {
    stuffs.get(i--).setSomething(true);
}
// I end up here because of live auto-updates

This is a problem, because the stuffs will no longer change. I had to do a search for -- in my code and fix all iteration like this.

The official workaround used to be this:

RealmResults<Stuff> stuffs = realm.where(Stuff.class).equalTo("something", false).findAll();
for(int i = stuffs.size()-1; i >= 0; i--) {
    stuffs.get(i).setSomething(true);
}
// I end up here because of normal iteration

This would still work fine in 0.89.0.

Since 0.89.0, this is valid code too (and in 3.0.0+, this automatically creates a snapshot collection):

RealmResults<Stuff> stuffs = realm.where(Stuff.class).equalTo("something", false).findAll();
for(Stuff stuff : stuffs) {
    stuff.setSomething(true);
}

The elements in the results still get invalidated though, but the results themselves don't change. (This is the same for snapshot collections as well in Realm 3.0.0+).

0.90.0:

Breaking changes

RealmChangeListener provides the changed object/Realm/collection as well (#1594).

All JSON methods on Realm now only wraps JSONException in RealmException. All other Exceptions are thrown as they are.

Marked all methods on RealmObject and all public classes final (#1594).

Removed BaseRealm from the public API.

Removed HandlerController from the public API.

Removed constructor of RealmAsyncTask from the public API (#1594).

RealmBaseAdapter has been moved to its own GitHub repository: https://github.com/realm/realm-android-adapters

File format of Realm files is changed. Files will be automatically upgraded but opening a Realm file with older versions of Realm is not possible.

So RealmBaseAdapter is now in realm-android-adapters, for 1.1.1 of Realm, use 1.3.0. Also adds RealmRecyclerViewAdapter. For 3.5.0, use 2.0.0 or newer.

RealmChangeListeners got an element parameter. Yay.

Also, Date now has milisecond precision.

0.91.0:

Breaking changes

Removed all @Deprecated methods.

Calling Realm.setAutoRefresh() or DynamicRealm.setAutoRefresh() from non-Looper thread throws IllegalStateException even if the autoRefresh is false (#2820).

Deprecated a lot of methods in 0.90.0, so

Breaking Changes:

Realm.allObjects*(). Use Realm.where(clazz).findAll*() instead.

Realm.distinct*(). Use Realm.where(clazz).distinct*() instead.

DynamicRealm.allObjects*(). Use DynamicRealm.where(className).findAll*() instead.

DynamicRealm.distinct*(). Use DynamicRealm.where(className).distinct*() instead.

Realm.allObjectsSorted(field, sort, field, sort, field, sort). Use RealmQuery.findAllSorted(field[], sort[])` instead.

RealmQuery.findAllSorted(field, sort, field, sort, field, sort). Use RealmQuery.findAllSorted(field[], sort[])` instead.

RealmQuery.findAllSortedAsync(field, sort, field, sort, field, sort). Use RealmQuery.findAllSortedAsync(field[], sort[])` instead.

RealmConfiguration.setModules(). Use RealmConfiguration.modules() instead.

Realm.refresh() and DynamicRealm.refresh(). Use Realm.waitForChange()/stopWaitForChange() or DynamicRealm.waitForChange()/stopWaitForChange() instead.

waitForChange() doesn't really work as people would intend to use it, so here's a workaround for 1.1.1 to 3.1.4 though, should be used only on background threads. refresh() will be re-added in 3.2.0.

Also, at some point Realm.getInstance(Context) was removed, use Realm.getInstance(new RealmConfiguration.Builder(Context).build()) instead.

---

After that, 1.0.0 came, so it's pretty much just that.

By the way, in 1.1.0, insertOrUpdate() was added which is faster than copyToRealmOrUpdate(), and doesn't return a proxy.

---

2.0.2:

• Primary keys are immutable on managed objects, once it's set, it cannot be changed, and it throws an exception if you try. Also, use realm.createObject(clazz, primaryKeyValue) if you use createObject() to create objects.

• You must call Realm.init(Context) at some point.

• Configuration Builder no longer receives Context.

• armeabi is no longer supported. (only v7a and the others)

---

No breaking changes until 3.0.0, but a ton of bugfixes.

---

3.0.0:

RealmResults.distinct() returns a new RealmResults object instead of filtering on the original object (#2947).

RealmResults is auto-updated continuously. Any transaction on the current thread which may have an impact on the order or elements of the RealmResults will change the RealmResults immediately instead of change it in the next event loop. The standard RealmResults.iterator() will continue to work as normal, which means that you can still delete or modify elements without impacting the iterator. The same is not true for simple for-loops. In some cases a simple for-loop will not work (https://realm.io/docs/java/3.0.0/api/io/realm/OrderedRealmCollection.html#loops), and you must use the new createSnapshot() method.

RealmChangeListener on RealmObject will now also be triggered when the object is deleted. Use RealmObject.isValid() to check this state(#3138). RealmObject.asObservable() will now emit the object when it is deleted. Use RealmObject.isValid() to check this state (#3138).

Removed deprecated classes Logger and AndroidLogger (#4050).

Due to the Realm ObjectStore Results integration, RealmResults is live again in transactions, just like back in 0.88.3 and before.

So simple for loops (indexing with for(int i = 0; ...) is prone to break - meaning you either need to reverse iterate them, or create a snapshot collection first.

OrderedRealmCollection<Thing> snapshot = results.createSnapshot();
for(int i = 0; i < snapshot.size(); i++) { ...

Also, RealmObject change listener will now also emit on deletion, you need to check for isValid() in the change listener. This is so that you can update the UI if the object has been deleted in the background.

3.1.0:

Updated file format of Realm files. Existing Realm files will automatically be migrated to the new format when they are opened, but older versions of Realm cannot open these files.

Nothing to do here, but it's worth a mention.

3.2.0-3.2.1:

Nothing to do here, except update proguard, because there was a bug introduced here. Added proguard section.

3.3.0: (and 3.3.1)

Nothing to do here, the bug was fixed which caused the Proguard problem in 3.2.0.

3.4.0:

Nothing to do here, although it's worth looking at the new @LinkingObjects API for inverse relationships.

In fact, it is recommended to replace bi-directional links with uni-directional link + inverse relationship.

3.5.0:

Breaking Changes

An IllegalStateException will be thrown if the given RealmModule doesn't include all required model classes (#3398).

If you haven't specified all RealmObjects in the modules() (in case you use multiple modules instead of just the default, for example RealmObjects from a library project), then you need to make sure you actually provide all RealmObjects that are part of the schema in your modules.

Previously it silently added them even if they were not in the modules, now that is not the case.

4.0.0:

Breaking Changes

The internal file format has been upgraded. Opening an older Realm will upgrade the file automatically, but older versions of Realm will no longer be able to read the file.

[ObjectServer] Updated protocol version to 22 which is only compatible with Realm Object Server >= 2.0.0.

[ObjectServer] Removed deprecated APIs SyncUser.retrieveUser() and SyncUser.retrieveUserAsync(). Use SyncUser.retrieveInfoForUser() and retrieveInfoForUserAsync() instead.

[ObjectServer] SyncUser.Callback now accepts a generic parameter indicating type of object returned when onSuccess is called.

[ObjectServer] Renamed SyncUser.getAccessToken to SyncUser.getRefreshToken.

[ObjectServer] Removed deprecated API SyncUser.getManagementRealm().

Calling distinct() on a sorted RealmResults no longer clears any sorting defined (#3503).

Relaxed upper bound of type parameter of RealmList, RealmQuery, RealmResults, RealmCollection, OrderedRealmCollection and OrderedRealmCollectionSnapshot.

Realm has upgraded its RxJava1 support to RxJava2 (#3497) Realm.asObservable() has been renamed to Realm.asFlowable(). RealmList.asObservable() has been renamed to RealmList.asFlowable(). RealmResults.asObservable() has been renamed to RealmResults.asFlowable(). RealmObject.asObservable() has been renamed to RealmObject.asFlowable(). RxObservableFactory now return RxJava2 types instead of RxJava1 types.

Removed deprecated APIs RealmSchema.close() and RealmObjectSchema.close(). Those don't have to be called anymore.

Removed deprecated API RealmResults.removeChangeListeners(). Use RealmResults.removeAllChangeListeners() instead.

Removed deprecated API RealmObject.removeChangeListeners(). Use RealmObject.removeAllChangeListeners() instead.

Removed UNSUPPORTED_TABLE, UNSUPPORTED_MIXED and UNSUPPORTED_DATE from RealmFieldType.

Removed deprecated API RealmResults.distinct()/RealmResults.distinctAsync(). Use RealmQuery.distinct()/RealmQuery.distinctAsync() instead.

RealmQuery.createQuery(Realm, Class), RealmQuery.createDynamicQuery(DynamicRealm, String), RealmQuery.createQueryFromResult(RealmResults) and RealmQuery.createQueryFromList(RealmList) have been removed. Use Realm.where(Class), DynamicRealm.where(String), RealmResults.where() and RealmList.where() instead.

So Rx1 support was replaced with Rx2 support, and removeChangeListeners() was renamed to removeAllChangeListeners().

Most other things only affect sync Realms, and from this point it is possible to use RealmList<String>, RealmList<Date>, and RealmList<Integer> as part of the Realm schema. Querying them is not yet supported, and they are not filled out by create*FromJson methods.

4.3.1:

Deprecated

RealmQuery.findAllSorted() and RealmQuery.findAllSortedAsync() variants in favor of predicate RealmQuery.sort().findAll().

RealmQuery.distinct() and RealmQuery.distinctAsync() variants in favor of predicate RealmQuery.distinctValues().findAll()

Instead of using realm.where(Blah.class).distinct("something") or realm.where(Blah.class).findAllSorted("something"), you can now do

realm.where(Blah.class)
     .distinctValues("something") // subject to change to `distinct()` 
     .sort("something") // hopefully will change to `sorted()`? // nope, it's `sort`
     .findAll();

5.0.0:

Renamed RealmQuery.distinctValues() to RealmQuery.distinct()

Removed previously deprecated RealmQuery.findAllSorted(), RealmQuery.findAllSortedAsync() RealmQuery.distinct() andRealmQuery.distinctAsync()`.

The OrderedCollectionChangeSet parameter in OrderedRealmCollectionChangeListener.onChange() is no longer nullable. Use changeSet.getState() instead (#5619).

So this means that realm.where(...).findAllSorted("field") should be realm.where(...).sort("field").findAll().

Also comes that OrderedRealmCollectionChangeListener used to send null as the initial change set, now that is no longer the case, and == null should be replaced with .getState() == OrderedCollectionChangeSet.State.INITIAL. This also means that you need to use realm-android-adapters 3.0.0 or newer with Realm 5.0+.

Also, if you relied on the names of __RealmProxy classes: they are named by their fully qualified name, including packages, like my_package_SomeObjectRealmProxy.

---

PROGUARD RULES

#realm older than 0.84.1
-keepnames public class * extends io.realm.RealmObject
-keep @io.realm.annotations.RealmModule class *
-keep class io.realm.** { *; }
-dontwarn javax.**
-dontwarn io.realm.**

#realm 0.84.1+ and older than 1.0.0
-keep class io.realm.annotations.RealmModule
-keep @io.realm.annotations.RealmModule class *
-keep class io.realm.internal.Keep
-keep @io.realm.internal.Keep class *
-dontwarn javax.**
-dontwarn io.realm.**

#realm 0.89.0+ and older than 1.0.0
-keep class io.realm.RealmCollection
-keep class io.realm.OrderedRealmCollection 

#realm 3.2.0 and 3.2.1
-keepnames public class * extends io.realm.RealmObject