High-quality data is necessary for the success of every data-driven company. It enables everything from reliable business logic to insightful decision-making and robust machine learning modeling. It is now the norm for tech companies to have a well-developed data platform. This makes it easy for engineers to generate, transform, store, and analyze data at the petabyte scale. As such, we have reached a point where the quantity of data is no longer a boundary. Yet this has come at the cost of quality. ... Poor data quality in Hive caused tainted experimentation metrics, inaccurate machine learning features, and flawed executive dashboards. These incidents were hard to troubleshoot, as we had no unified approach to assessing data quality and no centralized repository for results. This delay increased the difficulty and cost of data backfills. The lack of centralization in data quality also made the data discovery process inefficient, making it hard for data scientists and data engineers to identify trustworthy data.

As AI becomes more widespread, the question is whether programmers write code themselves or have chatbots write it. Customers usually expect quality. If AI can help deliver this quality faster, why not? Look, everyone knows that there is a programming language called Java. There are Apache Commons libraries. You can Google it, but can you do something with it? Can you bring value to the business? This is the point. LLM models are a tool, just like a library or a framework. However, it has other capabilities that need to be mastered and used to bring value. It will be a long time before AI can replace developers because there will always be something that needs to be fixed. Either it's an error in the code or something wrong with the configuration. For example, if a bot has already written code that seems to work, but an error appears. The developer can spend little time writing the code but later spends more time looking for the error. Let's take GitHub Copilot. Programmers note that the acceptance rate of suggestions from Copilot is up to 40%. 

“Great employee” is easy to type. It’s less easy to define. Here’s a short list to get you started. Scrub it by discussing the question with your leadership team. The habit of success: Some employees seemingly don’t know how to fail. Give them an assignment and they’ll figure out a way to get it done. Competence: As a general rule, it’s better to apologize for an employee’s bad manners than for their inability to do the work. Without competence, employees with a strong success habit can do a lot of damage by, for example, creating kludges instead of sustainable solutions. Followership: Leadership is a prized attribute for employees to have. Prized, that is, if they’re leading in their leader’s direction. Otherwise, if you and they are leading in different directions, all your prized leaders will do is generate conflict and confusion. Followership is what happens when they embrace the direction you’re setting and make it their own. Intellectual honesty: Some employees can be persuaded with evidence and logic. Others trust their guts instead. That’s a physiological error. You want people who digest with their intestines but think with their brains.

We need a collaborative approach with both disciplines. One cannot function properly without the other. For example, I cannot design multicloud-based systems that define different usages for different Cloud services on different clouds. ... Many assume that the engineering tasks are the easiest part of the journey to the cloud. After all, if the cloud architect is good, the configuration should work, and it’s just a matter of using sound AI tools to carry out deployment. Even worse, some companies are working just with engineers and hiring specific skills. The company may pick a cloud brand and hire security, application, data, and AI engineers in that cloud platform. They assume that this specific cloud platform is the correct and optimized platform, which will usually cause trouble. Oh, the solutions may work, but it could cost 10 times more to operate. Not surprisingly, these companies have an underoptimized architecture since they’ve given zero consideration to architecture or the use of cloud architects. AI won’t save you from needing a good architecture and a good set of engineering disciplines. 

New regulations springing up in various regions will be among the drivers of data center sustainability in the months ahead. There are two main groups of regulations emerging that affect data center operations, according to Jay Dietrich, research director of sustainability at Uptime Institute. One is financial reporting modeled on the Task Force for Climate-related Financial Disclosures (TCFD), which requires reporting on energy consumption and efficiency and associated greenhouse gas (GHG) emissions. The other is the European Energy Efficiency Directive (EED), which requires an energy management plan, an energy audit, and reporting of operational data. In addition, there are voluntary, country-specific standards and siting requirements for data center efficiency and operations in various countries around the world, Dietrich says. A current example of a TCFD-related regulation is the E.U. Corporate Sustainability Reporting Directive (CSRD), with reporting requirements rolling out from large to small enterprises beginning in 2025 and continuing until 2028.

Firms want their best people to stick around and give more of themselves. Studies have shown that improved employee collaboration and alignment with a common purpose is key to achieving that. But what is the best way to make that happen in the way we now wish to work and live our lives? Some suggest that the emergence of generative AI and new work tools can improve productivity regardless of the workplace setting. But perhaps a different, more human, approach is needed? The profound loosening of relationships that employees have with their firm and one another, requires a similarly fundamental reimagining of the role of the leader itself. Ultimately, this will not come through new technology, systems, processes, or HR policy (however well-crafted), but through the actions and behaviours of credible and engaging people managers. Firms need to re-establish a sense of cohesion and that needs people who are exceptional good at doing just that. Businesses can’t just issue ultimatums or mandates; they need a leadership approach that “coheres” employees to feel less remote from one another and the firm.

Prompt engineering is much more of a collaborative conversation than an exercise in programming. Although LLMs are certainly not sentient, they often communicate in a way that's similar to how you'd communicate with a co-worker or subordinate. When you're defining your problem statements and queries, you will often have to think outside the box. The picture you have in your head may not translate to the internal representation of the AI. You'll need to be able to think about a variety of conversational approaches and different gambits to get the results you want. ... While you might not necessarily be expected to write the full application code, you will provide far more value if you can write some code, test your prompts in the context of the apps you're building, run debug code, and overall be part of the interactive programming process. It will be much easier for a team to move forward if the prompt engineering occurs as an integral part of the process, rather than having to add it in and test it as a completely separate operation.

Isolating tenants with infrastructure has a higher initial cost, especially as you discover the right size for tenant workloads. Once you understand the cost for a tenant, it provides a very stable cost per tenant. Any unevenness in the cost profile represents a choice of timing. For example, if you use containers per tenant, you must decide when to commission your next cluster. Software-based multitenancy has an early advantage as it keeps the initial product price low. The marginal economics of onboarding a tenant are very low — almost zero. There comes a point when the initial design can no longer manage the load. The first port of call is vertical scaling — adding more power to the infrastructure to handle the load. This increases the cost per tenant but enables further tenants to be added. Eventually, you run out of vertical scaling options and look to horizontal scaling. This requires more investment as you need to handle load balancing, re-architect stateful interactions and introduce technologies such as shared cache.