Published inTowards Data Science·PinnedMember-onlyTop 25 Painful Data Migration Pitfalls You Wish You Had Learned SoonerLessons I wish I knew about before embarking on data migration journeys — "Life is like a box of chocolates; you never know what you're gonna get" — if only Tom Hanks knew about the perils of Data Migration (DM). …Data Migration11 min readData Migration11 min read
Published inGeek Culture·PinnedMember-only50 Data Engineering JargonAre you working in data engineering but tired of all the jargon and acronyms? Say no more, here is a list of 50 to get you started! — The world of Data Engineering has evolved significantly over the past decade. From a once humble data warehousing set-up to an entire discipline. This includes everything from data pipelines to building data lakes. The popularity of this field has increased, as has the jargon.Data9 min readData9 min read
Published inTowards Data Science·3 days agoMember-only3 Simple and Powerful Ways This AI Technique Will Transform Data ManagementAdvances in AI Will Continue to Disrupt the Modern Data Stack. — Large Language Models (LLM) have been all the rage lately, with the introduction of OpenAI’s ChatGPT. The majority of the 100m+ plus users are using Chat/Text Completions to make their daily lives easier. However, a little-known API provided by OpenAI called Embeddings is truly transformational in how we do search…Data4 min readData4 min read
Published inGeek Culture·Apr 4Member-only5 Powerful Ways Active Metadata Can Enhance Your Data Governance StrategyToo much data and not enough governance — let’s change that — Active Metadata is the key to unlocking the full potential of your data governance efforts. Big Data seems like an old term — and it has not stopped increasing in volume. Old methods are not good enough to handle this exponential growth. One method is using Metadata to drive critical…Data Governance4 min readData Governance4 min read
Published inTowards Data Science·Mar 6Member-onlyThe 7 Unusual Data Observability Use Cases to Improve Your Data GovernanceEnhancing Data Governance Is the Top Most Priority for Organisations — Data Observability is all the rage in the industry right now. Vast amounts of data and endless Data Quality issues have led us to this stage. Gartner sees it in the "innovation trigger" phase and has at least 5–10 years' worth of growth left before it plateaus. …Data Observability5 min readData Observability5 min read
Feb 4Member-onlyThe 5 Powerful Ways Data Marketplace Will Change the Industry5 Ways Data Marketplaces are Changing the Game with their Dynamic Impact — Data Marketplace is a modern Bazaar. It’s a way organisations can consolidate and share data internally and externally. Over the years, billions have been spent on modernising the legacy data stack. However, the self-serve data dream continues to be out-paced by large budget over-spend and poor data transformation. There are…Data5 min readData5 min read
Published inTowards Data Science·Jan 23Member-onlyThe 4 Small but Powerful Ways to Improve Your Data Skills This YearLevel up Your Data Game by Mastering These 4 Skills — A colleague recently asked how they can level up their data skills this year. Data skills can come in many forms. From Data Engineering, Analysis to Data Science & Visualisation. However, they all have a common goal: to drive business value. …Data4 min readData4 min read
Published inTowards Data Science·Oct 17, 2022Member-only5 Most Important Things to Include in a Modern Data Quality FrameworkModernise Your Data Quality Framework by Including These Changes — Data Quality (DQ) continues to be a big challenge for many organisations, especially those trying to modernise their data stack. Years of underinvestment in data programmes are now costing companies nail-biting millions of pounds in regulatory fines. But that’s the stick — what is the carrot? How can you convince…Data5 min readData5 min read
Published inTowards Data Science·Sep 6, 2022Member-onlyTop 4 Data-Quality Mistakes to Avoid on Your Legacy Data StackIn a rush for the modern data stack, don’t forget about legacy — I have read and written many Modern Data Stack (MDS) articles, and I am so glad that, as an industry, we are taking the challenge of Big Data with its even bigger problems head-on. …Data Quality4 min readData Quality4 min read
Published inTowards Data Science·Aug 8, 2022Member-onlyTop 10 Most Powerful Lessons I Learned by Delivering Data ProjectsCore Lessons That Will Ultimately Help You in Your Learning Journey — At the start of any new project/venture, you should reflect on the past and the lessons learned. Having delivered multiple complex data warehouses, data lakes and analytics projects, I learned a thing or two about what to do and what to avoid. The key to any successful project is the…Data6 min readData6 min read