The trading market used for almost every Cities: Skylines interaction

In Cities: Skylines, as your city grows you'll notice people moving around the city - whether on foot, or in a vehicle. They move around for a number of purposes: residents look for jobs and families, tourists visit entertainment centres, and city services such as the fire and police departments will attend calls. There's even a supply chain where logs, coal, goods, etc, both raw and processed, are taken from one business to another (or exported!) as they get refined.

I wanted to find out more about how this system works, but I couldn't find much written up about it online. Let's do a deep dive and see what we can learn!

The basics

In a (simplified) stock market, there are "buy" and "sell" offers, with a middleman between the two to connect them together. Whenever you see a person, or vehicle, moving from one place to another in Cities Skylines, it's often due to a TransferOffer going through the TransferManager (our "market"). Buildings, vehicles, and cities can list offers to "buy"/"sell" certain things which, once they're matched with the other side of the deal, results in it being delivered or collected.

Offers get listed with a TransferReason, a priority and an amount. Priority is used as a way of ranking offers, and the amount is self-explanatory. The TransferReason can be something tangible like Oil, or Coal, for an industrial business, but it could also be Fire for a burning building, Crime if the police are needed, or even Partner for when a cim is looking for love. There are plenty of other types too: hearses, garbage collection, schools and shopping are just some of these.

The information in this post comes from reverse engineering Cities: Skylines and reading the decompiled code. The snippets are close to what's shipped with the game, with some renaming and removing of less relevant parts to make this post clearer. If you have a copy of the game, and want to take a look yourself, the Cities: Skylines Modding Guide has a great introduction.

Taking out the trash

Let's take a look at how garbage collection works to get an understanding of this transfer system from the outside before digging in. The CommonBuildingAI class has a method HandleCommonConsumption which gets called from the SimulationStep of buildings in the game in order to calculate electricity, garbage, water use, etc. When the building accumulates some garbage (this is just a number calculated from building type, level, district policies and a few other factors), if there is 200 garbage in total then a dice is rolled and there's a 20% chance of continuing the process to get the garbage collected.

As long as the dice roll passes, and the player has unlocked the Landfill Site, the building then checks amongst its "guest vehicles" (m_guestVehicles) to get the total available cargo space amongst those which can carry TransferReason.Garbage. This number is then subtracted from the total garbage accumulation (m_garbageBuffer) to see how much garbage still needs collecting. If this number is above 200 then AddOutgoingOffer is used to list a "sell offer" (the building is selling its garbage).

Singleton<TransferManager>.instance.AddOutgoingOffer( TransferManager.TransferReason.Garbage, new TransferManager.TransferOffer() { Priority = remaining / 1000, Building = buildingID, Position = data.m_position, Amount = 1 } );

Only a single "unit" of garbage is sold (hence the Amount of 1) but this clearly represents the concept - you can also see how Priority is used to prioritise those buildings which have more garbage accumulated.

Looking for trash to collect

With our garbage proudly listed for sale, we need to find a buyer - luckily, the city's landfill site is interested.

The AI for this building can be found in LandfillSiteAI. This class has a method ProduceGoods (called as part of the PlayerBuildingAI class's SimulationStep) which starts by checking amongst its "own vehicles" (m_ownVehicles) to see the total capacity of vehicles capable of carrying TransferReason.Garbage. Notice how this step checks amongst its own vehicles rather than its guest vehicles as in the previous step - this is because the "guest vehicles" before were actually garbage trucks belonging to the landfill site, en route to collect the rubbish.

There's some logic around garbage capacity etc, but if the landfill site is in a state to receive garbage it lists an incoming TransferOffer like so:

Singleton<TransferManager>.instance.AddIncomingOffer( TransferManager.TransferReason.Garbage, new TransferManager.TransferOffer() { Priority = 2 - ownedGarbageTruckCount, Building = buildingID, Position = buildingData.m_position, Amount = 1, Active = true } );

The priority here is set to 2 - ownedGarbageTruckCount. The effect of this is that offers coming from a landfill site with fewer owned garbage trucks will be given a higher priority (although I don't know why the feature is designed in that way). If this Priority is negative, it will be treated as 0 within the offer matching algorithm (more on this later).

How offers get stored

The TransferManager class stores incoming and outgoing offers separately, but the structures are the same for each. This chapter will talk about outgoing offers, but there's an "incoming" pair for all of this code.

All outgoing offers (for all TransferReasons) for are stored in a single massive array m_outgoingOffers (262,144 items). This array is effectively divided into "blocks", with each block having a fixed length of 256 items, and representing a specific combination of TransferReason and its priority. That is to say, there can be 256 offers stored for TransferReason.Fire with a priority of 0, and they will all be stored in the same place - just before the block containing the TransferReason.Fire offers at priority 1, and so on.

[ /* [block 0] 0 - 255: TransferReason.Garbage, Priority 0 */, /* [block 1] 256 - 511: TransferReason.Garbage, Priority 1 */, /* [block 2] 512 - 767: TransferReason.Garbage, Priority 2 */, /* ... */ /* [block 805] 206,080 - 206,335: TransferReason.PlanedTimber, Priority 5 */, /* [block 806] 206,336 - 206,591: TransferReason.PlanedTimber, Priority 6 */, /* ... */ ]

This pattern will repeat to cover all TransferReasons and priorities. The block # value can be found with (int) (reason * 8) + priority.

There are two other arrays, m_outgoingCount and m_outgoingAmount:

• m_outgoingCount is used to get the current number of offers for a given combination of TransferReason and priority, which is then used as an offset to find an empty space in a "block" of offers within the array - the index at which the new offer will be inserted.
• m_outgoingAmount stores the total volume of offers for a given TransferReason, but this doesn't seem to be used for much.

Offers will be added at the given priority, if there's space, otherwise the system will iterate down through the priorities until finding one with room.

public void AddOutgoingOffer(TransferReason reason, TransferOffer offer) { for (int priority = offer.Priority; priority >= 0; --priority) { int offerBlock = (int) (reason * 8) + priority; int offset = (int) this.m_outgoingCount[offerBlock]; if (offset < 256) { this.m_outgoingOffers[(offerBlock * 256) + offset] = offer; this.m_outgoingCount[offerBlock] = (ushort) (offset + 1); this.m_outgoingAmount[(int) reason] += offer.Amount; break; } } }

Matching offers together

Both the incoming and outgoing offers have been passed to the TransferManager class, so it's time to take a look and see what happens to match these two together and ensure the garbage gets collected. Each simulation step, the manager will match all offers for a single TransferReason. This reason is decided using the games current frame index (GetFrameReason) and is passed to the MatchOffers method.

First, the optimal distance for the given TransferReason is calculated. Each reason has a specific optimal distance, and any offer under this distance will be chosen. See section "How close is close enough?" at the end for more on this.

Starting with a priority p of 7 and working down to 0, the offers are matched in blocks. For a given block, the first incoming offer is matched, then the first outgoing, then the second incoming, and so on, until all offers in that block have been matched. The priority is then decreased a level and the next block is matched, and this process repeats down the priority levels until all offers for this TransferReason have been matched. The next simulation step it will be a different TransferReason.

As in the previous section, matching offers is done separately for incoming/outgoing but it's really the same process just backwards, so I'll only show the outgoing flow here.

The outgoing offer is retrieved using the block index and the i value that we're using to iterate through that block. Then, a lower priority bound is calculated as Mathf.Max(0, 2 - p); an offer with priority 7 will consider offers all the way down to priority 0, whereas an offer with priority 0 will only consider offers down to priority 2. This works well in practice, as the system allows offers with a higher priority to have more candidates for matching than offers with a lower priority.

TransferOffer outgoingOffer = m_outgoingOffers[(offerBlock * 256) + i]; int lowerPriorityBound = Mathf.Max(0, 2 - p);

The system then starts at p and works downwards to that lower bound. For each value of p, the system then works through that offer block (skipping any already-matched offers in the block). For each offer (that doesn't originate from the same place), the distance is calculated using SqrMagnitude, and then a distanceValue is calculated. If this value is higher than bestDistanceValue, then this offer is more optimal. If the offer is below the optimalDistance from the beginning, then no more incoming offers for this priority are considered, and the loop moves to the next priority down.

int bestPriority = -1, bestOfferIndex = -1; float bestDistanceValue = -1f; for (int pOther = p; p >= lowerPriorityBound; pOther--) { int otherBlock = (int)reason * 8 + pOther; int blockCount = m_incomingCount[otherBlock]; float pOther2 = (float)pOther + 0.1f; // used to give higher priorities a better chance of matching if (bestDistanceValue >= pOther2) { break; } // starts at `incomingIndex` because at this point in the block we will have already matched // some of the incoming offers, so those need to be skipped for (int otherIndex = incomingIndex; otherIndex < blockCount; otherIndex++) { TransferOffer other = m_incomingOffers[otherBlock * 256 + otherIndex]; if (other.m_object == outgoingOffer.m_object) { continue; } float offerDistance = Vector3.SqrMagnitude(other.Position - outgoingOffer.position); float distanceValue = distanceMultiplier >= 0.0 ? (pOther2 / (1f + offerDistance * distanceMultiplier)) : (pOther2 - pOther2 / (1f - offerDistance * distanceMultiplier)); if (distanceValue > bestDistanceValue) { bestPriority = pOther; bestOfferIndex = otherIndex; bestDistanceValue = distanceValue; // if offer is within distance then store it and move onto the next priority if (offerDistance < optimalDistance) { break; } } } }

Once an offer is found, the purchase amount gets calculated (Mathf.Min(outgoingOffer.Amount, incomingOffer.Amount)) and StartTransfer is called for the offer pair. This loop then continues until the outgoing offer is completely fulfilled (or until no more matching incoming offers can be found) - then the system moves onto the next offer.

Sending a truck

The StartTransfer method in TransferManager is then responsible for initiating the transfer between a pair of offers. A single vehicle, citizen or building is chosen from the pair in order to handle the transfer. If either offer has a Vehicle, it's used (incoming offer is chosen over outgoing) - otherwise, if either offer has a Citizen, then it gets used (incoming chosen over outgoing again). Lastly, the Building is chosen - interestingly, in this case, the outgoing offer is responsible for the transfer rather than the incoming one.

private void StartTransfer(TransferReason reason, TransferOffer offerOut, TransferOffer offerIn, int delta) { if (offerIn.Active && offerIn.Vehicle != 0) { // retrieve vehicle from incoming offer and transfer Array16<Vehicle> vehicles = Singleton<VehicleManager>.instance.m_vehicles; VehicleInfo info = vehicles.m_buffer[(int) offerIn.Vehicle].Info; info.m_vehicleAI.StartTransfer(offerIn.Vehicle, ref vehicles.m_buffer[(int) offerIn.Vehicle], reason, offerOut); } else if (offerOut.Active && offerOut.Vehicle != 0) { /* retrieve vehicle from outgoing offer and transfer */ } else if (offerIn.Active && offerIn.Citizen != 0) { /* retrieve citizen from incoming offer and transfer */ } else if (offerOut.Active && offerOut.Citizen != 0) { /* retrieve citizen from outgoing offer and transfer */ } else if (offerOut.Active && offerOut.Building != 0) { /* retrieve building from outgoing offer and transfer */ } else if (offerIn.Active && offerIn.Building != 0) { /* retrieve building from incoming offer and transfer */ } }

In our garbage collection example, both the incoming and outgoing offers have a Building linked (from the CommonBuildingAI and the LandfillSiteAI), so the method above will choose the LandfillSiteAI and call StartTransfer on it. The landfill site then uses GetRandomVehicleInfo from VehicleManager to get a VehicleInfo for the correct service and level, before spawning that vehicle and calling StartTransfer on the vehicle.

The last two parameters for CreateVehicle set the vehicle flags TransferToSource and TransferToTarget, respectively - the TransferToSource flag is applied here to bring the garbage back to the landfill.

bool vehicleCreated = Singleton<VehicleManager>.instance.CreateVehicle( out vehicle, ref Singleton<SimulationManager>.instance.m_randomizer, randomVehicleInfo, data.m_position, reason, true, false ); if (vehicleCreated) { randomVehicleInfo.m_vehicleAI.SetSource(vehicle, ref vehicles.m_buffer[(int) vehicle], buildingID); randomVehicleInfo.m_vehicleAI.StartTransfer(vehicle, ref vehicles.m_buffer[(int) vehicle], reason, offer); break; }

The truck will now be spawned at the landfill site and begin to drive towards the building to collect the trash.

Collecting the garbage and coming home

The truck is now comfortably driving to the building. CarAI (the base class of GarbageTruckAI) calls this.ArriveAtDestination (virtual in VehicleAI and implemented in GarbageTruckAI) from its SimulationStep. ArriveAtDestination then calls ArriveAtTarget (or ArriveAtSource, but here it's arriving at the target)

The truck then calculates how much garbage it needs to collect (it can also drop off garbage at the target if transferToTarget, the last parameter in CreateVehicle, was true) and uses the BuildingManager to reduce the garbage at the building. The truck also calculates how much garbage it's now carrying, as well as clearing the target.

BuildingAI building = buildingManager.m_buildings.m_buffer[(int) data.m_targetBuilding].Info.m_buildingAI building.ModifyMaterialBuffer( data.m_targetBuilding, ref instance.m_buildings.m_buffer[(int) data.m_targetBuilding], (TransferManager.TransferReason) data.m_transferType, ref amountDelta ); if (transferringToTarget) { data.m_transferSize = (ushort) Mathf.Clamp((int) data.m_transferSize - amountDelta, 0, (int) data.m_transferSize); } this.SetTarget(vehicleID, ref data, (ushort) 0);

SetTarget checks if the vehicle has the TransferToSource flag set, and if so, sets the flag GoingBack. The method StartPathFind is then called, which checks for the GoingBack flag and then starts the journey back to the landfill site.

When the truck arrives at the landfill, ArriveAtSource is called, which checks for the TransferToSource flag and uses the BuildingManager again to transfer the garbage material from the truck into the landfill.

The vehicle then gets released, marking the end of this round-trip. The garbage has been collected and deposited at the landfill site!

How close is close enough?

The optimal distance for a given TransferReason is calculated like so:

float distanceMultiplier = TransferManager.GetDistanceMultiplier(reason); float optimalDistance = (double) distanceMultiplier == 0.0 ? 0.0f : 0.01f / distanceMultiplier;

As you saw earlier, when considering offers, the first offer under optimalDistance is automatically chosen for a given priority. This distance is different for different reasons, so I graphed a few of the more common reasons in the game to show these distances.

Note that this value is not the maximum distance but rather a distance under which any option is chosen.

Conclusion

One of my favourite things to do in Cities: Skylines is, once my city has grown a bit, to watch the people within the city going about their daily life. It can be fascinating to see the huge variety of things going on - to see industrial buildings supplying other industrial buildings to eventually supply your shops, for people to then visit those shops, and, sometimes, even die in those shops, to be collected in a hearse. The organic nature of their journeys and activities keeps the game constantly fresh and exciting.

After looking into how this all pieces together, my fascination now runs even deeper. Colossal Order have done a fantastic job at creating a system so versatile and powerful that it can be used to drive such a large portion of the gameplay. There is support for up to 256 different TransferReasons, with 113 currently in use. It looks like around 60 of those come from DLC, so evidently their engineering efforts paid off!

I'm really looking forward to booting up Cities: Skylines again, this time with just a little more appreciation for how much complexity is involved in taking garbage from a house to a landfill site.

Thanks

Thanks, of course, to Colossal Order for making such an enticing game that I decompiled it (it's now part of a fairly short list of games), and also thanks to pcfantasy's MoreEffectiveTransfer mod for giving me a great starting point to understand this.