At the end of a Beta Project, we’d like to let people know how the trial went, what we measured, how feedback and suggestions were taken into account, and what the outcomes of the trial are going to be.
We’re calling this idea a ‘Beta Project Report Card’ and we’ve prepared two examples below.
If commenting on social media please add #ReportCardBeta to help us follow what you think of the idea, and all feedback and suggestions!
As the traffic light box artworks have just recently been adopted as a solution, and will be rolled out to 200 boxes in the city, below is an initial stab at how we might report to citizens what was the outcome of that trial, and on what basis that conclusion was reached.
This is a follow up to the 1% Metrics blog post where we discussed what a local authority might measure in order to assess whether a trial worked.
(As always, this is an early and dirty version so that we can, as always, hear your immediate thoughts and suggestions. For that reason, we’ve purposefully left them pretty basic – the below were simply done in Word and Powerpoint – so that it’s easier for you to feel that you can suggest changes or to comment.)
Beta Project Report Card – Summary Type
This is an example of the type of sign that we’ve printed off at A4 size, laminated, and put onto the 11 traffic light boxes where the trial took place.
These summary Report Cards serve two purposes:
- Many (probably most) people will simply be interested in a brief summary and what the outcome of a trial was…and importantly, what’s next! This summarises a trial for those people, and if they’d like to find out more, they can choose to read the more-detailed report below.
- Not everyone is on social media, or uses email as a regular communication tool. As a City Council, we need to be including as wide a range of citizens as possible…so the idea is that these might be printed off and put up at the location where the trial took place, and so a summary format is best.
Beta Project Report Card – More Detailed
This is an example of what a more detailed ‘Beta Project Report Card’ might be like. It generally would only be available online (though some copies might be printed off in certain situations…but the ‘default’ would not be to spending time and money on prints).
They present the outcome as one of 4 possible recommendations:
- Solution would appear to work. Push to Consent stage to begin to develop the necessary corporate policy.
- Need for further learning. Persevere, and develop another iteration of the trial.
- Need to reconsider the initial aim and direction. Pivot, using learnings gained to date.
- Solution does not appear to work. Pull it, and look for other solutions.
When should Report Cards be issued?
With regard to the traffic light box artworks Beta Project there was a period when the trial had finished and been ‘removed’ (the boxes repainted grey), but it hadn’t yet formally been given the go-ahead (planning, arts office, traffic department, area offices, and so on)…so people wondered about what was the status of the project.
It threw up the question for us as to whether a Report Card should be done…
- Immediately at the end of the trial and irrespective of the outcome…
- Would be at a stage when some technicality might still crop up which could prevent it going ahead, but would be sooner for people and when the trial was fresher in their minds.
- We wouldn’t yet know what the next steps would be, so would need to tell people that separately.
- This would make most sense in terms of some projects requiring further trialling as we’d simply announce our measurements and why we think it needs another trial iteration to learn more.
- Or whether we should hold off until we had a complete update for people…
- So wouldn’t be giving potentially false hope.
- And could announce the next steps at the same time.
We’d love to hear your feedback and suggestions please!
- What do you like / not like about these?
- What would you like to see more of / less of?
- What changes would you suggest?
- When do you think we should issue them?
- We’re calling them ‘Beta Project Report Cards’ as a working title. What do you think…would you suggest a different name?
- Any other thoughts?
(It’s worth noting here, that the traffic light box artworks was the very first Beta Project, so many of the ‘measurements’ in these report cards are often of the more qualitative rather than quantitative type. We’ll also blog our more recent thoughts on this aspect soon.)
Regarding the ‘When should Report Cards be issued?’ section above, the current method of including the ‘What’s Next’ within the Report Card isn’t quite working – there’s too much of a lag between the end of the trial and issuing the Report Card.
So Report Cards will now reflect method 1 above, and will be issued very quickly after the end of the trial and will simply give ‘the results’, rather than ‘the results + the decision’. Ie they will separate the reporting aspect from the decision making aspect.
As the Beta Projects concept is becoming more known, the number of people giving feedback is also greatly increasing. Rather than list all comments (as we usually have to date), future Report Cards could just give a few as samples, and link to the remainder in the cloud (such as a Google Doc) in order to maintain transparency and to reduce natural bias which could occur when only including a selection of all feedback received.