Where is the data context in Dialog Flow (API.ai)

I have been reading about Dialog Flow and there is one thing that is still unclear for me. I'll try to give an example.

I want to implement a conversion as following:

User: Hello Google, what are some interesting cities?
Bot:  Hello there! Sydney, New York and Berlin are nice.
User: Could you tell more about the second city?
Bot:  Sure. New York is amazing. In New York, you can ...

As you see, I am building a data context. After the first question, we should remember that we answered Sydney, New York and Berlin, so we understand what the second city actually means in the second question.

Should we store this data in the webhook service or is this stored in a context in Dialog Flow? If we have to store such data in the webhook service, how can we distinguish between different ongoing conversations?

Storing it in a Dialogflow Context is an ideal solution - this is exactly what Contexts were made for! You phrased your question using the same term, and this is no coincidence.

Conceptually, you might do this with a setup like this:

User: What are some interesting cities?

Dialogflow sees no contexts and matches an Intent asking for cities.

Agent replies: Sydney, New York, and Berlin are nice.
Agent sets context "cities" with parameter "cities" -> "Sydney, New York, Berlin"

User: Tell me more about the second one?

Dialogflow has an Intent that expects an incoming context of "cities" with a text pattern like "Tell me more about the (number index) one?" It sends the request to that Intent along with the currently active contexts.

Agent get a parameter with the index and the context "cities". It looks up the parameter for it, turns the string into an array, and gets the city based on the index. 
Agent replies: New York is a fun place to visit!
Agent sets context "city" with parameter "current" -> "New York"

User: Tell me more!

Dialogflow matches this phrase and that the "city" context is still active and sends it to an event that reports more.

Agent says: More awesome stuff about New York.

User: Tell me about that first city instead.

Dialogflow matches it against the same intent as before.

Agent says: Sydney is pretty cool.
Agent changes the "city" context so the parameter "current" -> "Sydney" and "previous" -> "New York".

You can now create other intents that handle phrases like "Compare these two" or "tell me more about the other one".

Update

This setup strikes a good balance between what Dialogflow does well (parse messages and determine the current state of the conversation) and what your webhook does well (determine the best answers to those questions).

You could probably do much of that inside Dialogflow, but it would start to get very very messy very quickly. You would need to create multiple Intents to handle the results from each value individually, which doesn't scale. You'd also need to create a Context for each city (so you'd have a "city_ny" and "city_sydney" Context), since you can only match on the presence of a Context, not the parameters it might have.

Using the webhook (even the built-in fulfillment system that we now have) will likely work much better.