You are a traditional database administrator (DBA) and have been constantly overwhelmed by the craze in the data science field. You see data science articles making rounds all over the Internet. You question whether your expertise in traditional database administration will continue to hold its value and remain relevant in the future. Well, hang on. Just because it has taken the data world by storm doesn’t necessarily mean you should run after it. Remember the proverb, one man’s meat is another man’s poison? What is suitable for one is not necessarily so for someone else. The same holds in this beautiful field of data. It’s a team sport. We need to go beyond the buzzwords and understand the role definitions correctly and how they inter-operate.
While it’s true that the latest advancements in Artificial Intelligence do present innovative methods to solve business analytical problems, those areas are not the only ones that make up the data ecosystem in an organization. All data-related positions go hand-in-hand and are all interdependent. This interoperability among different data positions makes each position unique and irreplaceable. So let us not go overboard with the fancy job titles.
Some of the critical teams in the realm of data include software engineering, business intelligence, database administration, data engineering, data analysis, and data science. Each of these teams performs specific roles in creating value from data. The mistaken understanding that most of their job responsibilities overlap leads to team failures and overall project failures. Each team depends on the other to function correctly and efficiently.
While every organization is different, typically, business intelligence differs from data science in that teams do not do a lot of coding. Likewise, data warehousing differs from data engineering in that data engineers must have programming language and work with distributed systems. A database administrator’s role generally falls anywhere between a data engineer and a data analyst or a data scientist. If the DBA team is missing, the burden passes to the data engineers who aren’t as capable of doing it. For example, a DBA knows various performance tuning and query optimization techniques that a data engineer may not know. The absence of DBAs means data engineers take matters into their own hands, potentially leading to bad query practices and poor performance. These are very costly to fix in production.
A DBA is also responsible for enforcing database security and ensuring the data access adheres to the organization’s policies. A DBA job is incomplete without ensuring proper high availability and disaster recovery for mission-critical database systems.
If data engineers perform DBA tasks, it won’t be without cutting corners on the robustness of the data products. DBAs have spent their entire careers dealing with database management and administration, which data engineers generally lack. With almost all data held in database systems and exposed with a SQL interface, it is impossible to imagine productionizing data products without SQL-focused DBAs.
It is essential to understand the distinction between data engineers and DBAs. While both positions go hand-in-hand, they are not the same. Data engineering teams are programming-focused with distributed systems skills. They write code and build data pipelines from scratch using disparate open-source technologies. They are not known to perform database backups, query optimization, or enforce security, which is all where a DBA’s role shines. Though DBAs may not necessarily work in distributed environments, they work with already mature technologies and tools, unlike data engineers who may be required to experiment with tools not mature enough. DBAs determine the level of access data engineers need to run their processes.
Can DBAs fill in data science roles? DBAs’ lack of programming knowledge creates a barrier that can only be overcome with heavy training. However, data analysts or business intelligence teams can transition into data science with proper retraining.
All that said, the DBA team is facing significant changes with the advent of new data roles and responsibilities. With data growing globally at an unrealistically high rate, organizations lean towards database teams that possess the cutting-edge skills required in today’s big data ecosystem rather than the traditional data warehouse or DBA teams.
DBA jobs are not going away entirely, but if you don’t keep yourself updated with the latest versions of the database system on-prem or cloud infrastructure, your involvement in the data projects risks fading away. Therefore, while it’s true that without DBAs, organizations risk abusing their databases by running poorly designed queries, it is equally crucial for modern DBAs to learn new skills and be prepared for the ever-changing data field.
So, congratulate and pat yourselves on the back for being part of the excellent data team and playing a significant role in contributing to your company’s interest.
