Building big things with lots of people who don't work the same way

Let's assume that we're building the right thing.

In these cases, the actual user-exposed behaviour tends to be quite trivial while the bulk of the project is dealing with behind-the-scenes systems.

Let's assume that the project consists of multiple teams from multiple vendors who work in different ways.

One way we might respond to this would be to encourage or even attempt to force the teams to work together in the same way.  This is generally naive and ineffective.

A more typical response would be to work out how to separate the work out into independent packages / components / services.  This reduces communication overhead and helps isolate differences in work practices.

How would we determine the interfaces?
The typical approach to determine components and interfaces is to get an enterprise architect or bunch of architects together and get them to work it out.

The extent to which this approach is effective is highly dependant on whether the architects have previously built multiple similar systems.  With large, complex systems, this familiarity is highly unlikely.

Instead, a safer approach is Conquer and Divide.  Start with a smaller, combined team, and build the core of the overall system together.  This should flesh out what actually makes sense in terms of separate components and interfaces.  Once that is clearer, then the separate teams can split off appropriately.

Some of the biggest risks are about integration and schedule collapse due to dependencies
With a large, complex system built by multiple teams from multiple vendors, who work in different ways, by far the largest risks are related to separate components built by separate teams not working together, and schedule collapse due to problems with coordinating dependencies across teams.  Again, this assumes that we're building the right thing overall.

How might we get early detection of integration problems?
My preference is to use automated interface / contract tests to explicitly capture behaviour expected from all teams.  Both sides of an interface will know what they should expect and what is expected of them even if components have not yet been built yet.  If an implementation problem requires a change in the interface behaviour, then we explicitly know that we must inform other teams.

How might we prevent schedule collapse?
If schedules are important then it is worth buying the insurance of building multiple options for critical components, that is .  That is, implement a simple version first and then implement a more complicated but better version after.  Because the simple version is built first, we have already protected ourselves from the scenario where another team needs to integrate with our component and we have nothing ready.