What exactly is considered a breaking change to a

Rust crates use Semantic Versioning. As a consequence, each release with a breaking change should result in a major version bump. A breaking change is commonly considered something that may break downstream crates (code the depends on the library in question).

However, in Rust a whole lot has the potential of breaking downstream crates. For example, changing (including merely adding to) the set of public symbols is possibly a breaking change, because downstream crates can use glob-imports (use foo::*;) to pull symbols of our library into their namespace. Thus, adding symbols can break dependent crates as well; see this example.

Similarly, changing (adding or changing the version) the set of our dependencies can break downstream builds. You can also imagine that the downstream crate relies on a specific size of one of our public types. This is rarely, if at all, useful; I just want to show: everything could be a breaking change, if only the downstream crate tries hard enough.

Is there any guideline about this? What exactly is considered a breaking change and what not (because it's considered "the user's fault")?

There is a Rust RFC on this subject: RFC 1105: API Evolution. It's applicable to any Rust library project, and it covers all kinds of changes (not just breaking changes) and how they impact semantic versioning. I'll try to summarize the key points from the RFC in order to not make this answer a link-only answer. :)

The RFC acknowledges that pretty much any change to a library can cause a client to suddenly stop compiling. As such, it defines a set of major changes, which require a bump of the major version number, and a set of minor changes, which require a bump of the minor version number; not all breaking changes are major changes.

The key attribute of a minor change is that there must be a way that clients can avoid the breakage in advance by altering slightly their source code (e.g. change a glob import to a non-glob import, disambiguate an ambiguous call with UFCS, etc.) in such a way that the code is compatible with the version prior to the change and with the version that includes the change (assuming that it's a minor release). A minor change must also not force downstream crates to make major breaking changes in order to resolve the breakage.

The major changes defined in the RFC (as of commit 721f2d74) are:

Switching your project from being compatible with the stable compiler to only being compatible with the nightly compiler.
Renaming, moving or removing any public item in a module.
Adding a private field to a struct when all current fields are public.
Adding a public field to a struct that has no private fields.
Adding new variants to an enum.
Adding new fields to an enum variant.
Adding a non-defaulted item to a trait.
Any non-trivial change to the signature of a trait item.
Implementing a fundamental trait on an existing type.
Tightening bounds on an existing type parameter.
Adding or removing arguments to a function.
Any other breaking change that is not listed as a minor change in the RFC.

The minor changes defined in the RFC (as of commit 721f2d74, breaking unless specified) are:

Altering the use of Cargo features on a crate.
Adding new public items in a module.
Adding or removing private fields in a struct when at least one already exists (before and after the change) [not breaking].
Turning a tuple struct with all private fields (with at least one field) into a normal struct, or vice versa.
Adding a defaulted item to a trait.
Adding a defaulted type parameter to a trait [not breaking].
Implementing any non-fundamental trait on an existing type.
Adding any item to an inherent impl.
Changing an undocumented behavior of a function.
Loosening bounds on an existing type parameter [not breaking].
Adding defaulted type parameters to a type or trait [not breaking].
Generalizing an existing struct or enum field by replacing its type by a new type parameter that defaults to the previous type [breaking until issue 27336 is fixed].
Introducing a new type parameter to an existing function.
Generalizing a parameter or the return type of an existing function by replacing the type by a new type parameter that can be instantiated to the previous type.
Introducing new lint warnings/errors.