Give every vendor the same scenario and sample data. Ask them to process a partial order with a price change after approval, or use another exception your team handles regularly.
Watch how each vendor handles exceptions
Ask who can change the transaction, what the system records, and how users can reverse a mistake. Record which requirements work as delivered and which need configuration or custom development.
Agree on criteria before the demo
Separate mandatory requirements from preferences. Have business and IT participants record observations independently, then reconcile their scores against what the vendor demonstrated.
Illustrative photo by Dylan Gillis on Unsplash.