Leveraging Cloud Transformation to Build a DevOps Culture | AWS Public Sector Summit 2016

Editor's Notes

1. Of course there are many situations where this may not be realistic, but remember that Conway’s Law talks of the “communication structures” of an organization rather than reporting structures. There are often opportunities to improve communication pathways in lightweight ways without having to grapple with thornier organizational issues.
2. Forrester writes that AWS is the best fit for the DevOps pro segment
3. Nowadays, there are lots of little decisions made every day by many different people that affect your bill. You may have 150 developers each with the power to provision a server. The good news here is that everyone can get the resources they need in order to move more quickly and you don’t have to buy ahead of demand. The challenge is that the spending power is now decentralized and there’s less control over who can spend money.
4. The tools for efficiency need to be from the right era. In an older car you had the ability to record your mileage, then look at how much gas you put in and calculate your miles per gallon then try to improve your driving efficiency . The problem is nobody did it because it took too much time and the gap between actions and results was too wide. So most of us just drove and didn’t think about it. Now you’ve got dashboards like this one that tell you—as you drive—how the weight of your foot affects your MPG. If you’ve ever driven a Prius, it’s amazing how you automatically start making changes to your driving style. Not just to save money on gas but it becomes a game to see how efficient you can be. Efficiency becomes a habit over time. Take that power of one person having this power, then multiply it out across everyone in your organization. That is a culture of cost management.
5. It’s no longer one person’s responsibility. There are now a lot of players making decisions about what to buy and needing visibility into spending. But they need tools to do this.
6. secure innovation at speed and scale
7. secure innovation at speed and scale
8. They also added a new person. Someone dubbed the RI Czar. This is a person or team focused on looking at the AWS billing data each month to identify opportunities to increase Reserved Instance coverage. In small companies this is a single person, in larger ones it can be a small team. This person acts as an unbiased 3rd party liaison between the various stakeholders in the company to ensure the right data is being analyzed, the right conversations are being had and ultimately the right purchases and modifications are being made. They made this someone’s job. The person was measured on the results and ran the process we just laid. Is this person full-time? They can be. Usually not, but the numbers actually support dedicating a material amount of an FTE’s time to the process. - Proper purchasing can save 30-60% of your EC2 !!! Bill. - Assuming a $1M spend on EC2, that’s a lot of savings…enough to pay for some committing someone’s time to the process Interestingly, this person generally doesn’t sit in the technology organization. We’ve seen it be a technically minded person in the finance, procurement or vendor management organizations. Remember Ris are a billing construct (like a coupon) that is applied after the technology decision is made.
9. So…how do you build the culture. Put data in the hands of the people (not once a monthly or whenever there’s a problem) but every day. Enact policies and evangelize best practices (remember that running at scale on cloud is new to most companies and they simply don’t know where to start) Incentivize good behavior (in fact, the most effective companies gamily and openly share how each group is doing)
10. The first stage “Cost Visibility” can be summed up as “What gets measured gets improved.
11. Tips in Stage I: Get each stakeholder the spending fundamentals daily. Specifically get them visibility into their portion of the bill so that they can become accountably for daily changes. Let each team see other’s spending habits. This helps create social pressure and incentivizes good behavior.
 Create broadly available dashboards. Share them widely and put them up in shared workspaces.
12. Cloudability offers some powerful ways to deliver cost visibility to everyone. There’s a completely widgetized dashboard system that lets you view your spending in any way you want…look at costs by tag overlaid with usage data, monitor RI utilization compared to instance counts, whatever you like in a system that can be configured to work for execs as well as engineers. This type of visibility is the first step toward creating your culture.
13. OK let’s talk about the next stage, allocation is all about using the appropriate tools to split out the bill to answer the question of where did the dollars go?
14. These are your tools. Here’s a structure we’ve seen work well. Tags are highly flexible but 100% coverage is difficult due to compliance and the fact that not all charges can be tagged.
 Linked accounts offer clean chargeback but limit reporting options. Most commonly we see eng/ops folks push for tag-based solutions while finance folks tend to prefer account-based divisions due to the clean reporting lines they create. Solution: Use Both with linked account splitting out most important divisions —- Linked accounts to split financial data Tags to split operational data 

15. Tips for allocating costs: - Get consensus on the taxonomy. But we’ve found it works best for Finance to drive the conversation. - Define 2-3 mandatory tags that must be applied before resources are deployed Tagging on deployment is key, we’ve seen far too many post-deployment tagging automation schemes not work. - Consider adding a Tag or Terminate rule to enforce tagging.
16. Efficiency. This Stage is all about turning off waste, resizing and rightsizing instances.
17. Here’s what’s important to cover in this stage. Make sure each team is getting weekly reports on their underutilized instances, you can schedule this in Cloudability. Gamify clean-up by creating a custom leaderboard that everyone can see. Again, check out our new dash boarding features for this. Do a monthly whole company waste review. Work with your cost management provider, AWS and internal thought leaders to do a monthly review of where you could do better.
18. Everyone’s favorite stage - Savings through lowering hourly costs.
19. You reservations and infrastructure function as two sides of the same coin.Your infrastructure is fluid and elastic as such you need to keep modifying and adjusting your reserved instance portfolio to match. Sunk cost + on-demand
20. Ok, now we get into the nirvana stage of cloud cost management. Unit Cost.
21. You should be watching unit cost and it’s affect on margin and managing your business to that number. If you’re successful, your bill may always go up. Focusing on unit cost will give you visibility into how that ties into the business—and get your management team off your back.