Tobias Oberstein

💛 working with TMDL on Direct Lake I tested my personal new highlight feature of this month today with Marcus Wegener and Andrea Hupp - it works on all explicitly created Semantic Models - the "edit mode" has to no report - like in the Web-Service. But the DAX query view feels "more close" - all changes are synched to the published model. No saving needed. 🤯 but the real game changer is to export the Power BI file a project, then I can: - edit everything in TMDL code and sync the changes fast to the published model (but also beeing able to refuse the changes). Deployment never have been so easy 🤩 - as an empty report appears when I open the .pbip file, I am able to test my changes in real visuals. Even when I love the DAX query view... sometimes visuals show me more insights when I investigate some data issues 😉 👾 Doing the Video was also a chance to use the AI Dubbing one more time - it still feels a little strange, but I still like it and its faster doing the recording multiple times. I still sound different in real english tutorials I do from time to time 😅 Looking forward for more news from the European Microsoft Fabric Community Conference ..! 😃 #powerbi #fabric #update

18

2 Kommentare

🧠 The myth: More data makes an #AI model smarter and therefore more accurate 💪 The truth: It’s not just about data—it’s about #parameters With the around OpenAI’s #GPT4o it might seem that feeding more data into these models makes them smarter, the real game-changer is the number of parameters they have like Meta #Llama 3.1 with 450b parameters 💡 More parameters mean these models can process and understand data in more sophisticated ways, leading to increased accuracy and the appearance of being “smarter.” More parameters = higher trustworthiness, even with more focused data like the proprietary #data in your organization In the world of conversational user interfaces becoming the norm, more parameters will be crucial for a seamless experience. While yet too slow and too expensive for real fast interactions between humans and machines, with #quantumcomputing we will move in a new area of responsible #UI #design #AI #MachineLearning #LLM #conversationaldesign #hmi #Innovation

5

1 Kommentar

Prithvi WxC Released by IBM and NASA: A 2.3 Billion Parameter Foundation Model for Weather and Climate Researchers from IBM Research and NASA have introduced Prithvi WxC, a 2.3 billion parameter foundation model for weather and climate forecasting. The Prithvi WxC model incorporates 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2), a high-resolution dataset covering global atmospheric conditions. This model employs a state-of-the-art encoder-decoder transformer-based architecture, allowing it to capture local and global dependencies in the atmospheric data efficiently. Using a transformer model facilitates handling long-range dependencies in the data, making it possible to model complex atmospheric interactions at various scales, from local to global. Prithvi WxC’s core architecture features a combination of local and global attention mechanisms that enable it to process large token counts, effectively capturing spatial and temporal patterns in the input data. It also employs a mixed objective function that integrates masked reconstruction and forecasting tasks. This unique approach allows the model to generalize well across different applications, ranging from autoregressive rollout forecasting to estimating extreme weather events. Also, the model incorporates a pretraining phase with 25 encoder and 5 decoder blocks, utilizing advanced AI techniques such as masked autoencoding and variable lead-time prediction. The model’s flexibility is further enhanced by its ability to incorporate additional tokens from off-grid measurements during fine-tuning, making it adaptable for various downstream applications.... Read our full Article on Prithvi WxC: https://lnkd.in/gfdgp_5u Paper: https://lnkd.in/g4re9tJm Model on Hugging Face: https://lnkd.in/g7peMGWG GitHub Page: https://lnkd.in/grM6jxaj IBM IBM Research NASA - National Aeronautics and Space Administration Johannes Schmude Will Trojak Johannes Jakubik Daniel Salles Chevitarese Julian Kuehnert Romeo Kienzler Daniela Szwarcman

36

2 Kommentare

What Karl says. There's going to be two amazing talks about Hopsworks based #ai and #ml systems: * Build a personalized Bitcoin (BTC) virtual assistant in Python with Hopsworks and LLM function calling by Javier de la Rúa Martínez (https://lnkd.in/eQfWpFRU ) * Build TikTok's Personalized Real-Time Recommendation System in Python with Hopsworks by Jim Dowling (https://lnkd.in/eXEHePjG) If you are in the #berlin area next week - go and take a look!

7

💡 The 2024 🇨🇭 Open Source Study confirms that 97% of Swiss corporations are using #opensource software …. and 40% are already using #opensource #ai tools and models. Here my Top 5 takeaways from the study ⬇ 1. There are already 700’000+ open source #ai models on Hugging Face 2. #TensorFlow and #PyTorch are the top used #ai open source tools. 3. #Llama and #Mistral are by far the most used open source #ai models. 4. #opensource is used where nobody can really see it! None of the #cloud environments we’re using in private and professional live would work without Open Source! 5. #linux, #docker and #kubernetes are key technologies to run today’s Cloud environments. A lot of interesting articles about the economics of #opensource and real world examples are complementing the study. Thanks to CH Open, swissICT and Berner Fachhochschule BFH for running an providing this interesting study! Here you can find the details (in German): https://oss-studie.ch/ Matthias Stürmer Alain Gut Christian Keller Richard Zobrist Katharina Göppinger Lise Crettaz Philipp Spaeti

33

4 Kommentare

This slide by Maja Miličić Brandt and Steffen Lamparter of Siemens also describes the difference between #RDF (left) and #GraphQL (right) that I experienced when designing Ontotext Platform's #SemanticObjects. Slide from https://lnkd.in/d_cD5MKa

39

4 Kommentare

#PolyglotModeling: https://lnkd.in/e2u2xtmX discusses various approaches for a simple technology-independent modeling formalism, from which you can generate a variety of technical artefacts (#ontologies, #SHACL shapes, #JSONSchema, #GraphQL schema and queries, etc). This relates to a knowledge-centric (#semantic) vs data/object-centric (developer-friendly) dilemma as listed at https://lnkd.in/e8ynvyEe.

12

1 Kommentar

Granite Code released! 🧑🏻‍💻 @IBM just released a family of 8 new open Code LLMs from 3B to 34B parameters trained on 116 programming languages and released under Apache 2.0. 🔥 Granite 8B outperforms other open LLMs like CodeGemma or Mistral on benchmarks and supposedly supports COBOL! 🪨🤯 TL;DR: 🧮 8 models (base + instruct) from 3 to 34B parameters 🦙 Based on the Llama architecture 🪟 Context from 2k (3B) to 8k (20B+) models 🌐 Trained with @BigCodeProject Stack and Github on 116 programming languages 🔠 2 Phase pertaining code only (~4T tokens) then high-quality code + language (500B tokens) 🛫 34B model is depth upscaled (merging) from 20B and further trained 🧠 Trained on IBM Vela and Blue Vela supercomputer 🔓 Released under Apache 2.0 🤗 Available on @huggingface ❌ Cleaned and filtered Datasets not released ❌ No mention of decontamination in the paper Paper: github.com/ibm-granite/gr… Models: https://lnkd.in/eiZXbeQH

99

GraphRAG Microsoft recently open-sourced GraphRAG, https://lnkd.in/en_Qa5hU, a tool to develop pipelines to extract knowledge graphs from raw text. A decade ago, I worked on a project with Vineel Gujjar to construct knowledge graphs from aircraft technician notes. A cascaded design involved a system extracting entities such as "issues," "parts," and "fixes" and encoding their relationship in a knowledge graph to ultimately figure out the best "fix" for an "issue" involving a "part." LLM-based NLP systems augmented with GraphRAG approach would have made it much easier to rapidly develop and deploy complex systems like the one we developed a decade ago. Ramasubramanian (Ramsu) Sundararajan Debasis Bal

34

2 Kommentare

Defining the Meaning of Your Data – A Practitioner View Last Thursday, we had what could be described as the most exclusive data event of the year — the Semantic Layer Symposium. It brought together some of the most renowned data professionals, all gathered in a spectacular setting — the private library of the "Bayerische Hof Hotel” in Munich. Surrounded by hundreds of books, we dived deep into the heart of a essential topic: how to make data more meaningful, interconnected, and accessible to the people who use it. Fittingly, the profession of librarians — now being redefined by AI — served as an inspiration for this work. But that’s a story for another post. I had the privilege of joining the panel alongside Helen Lippell and Daniel Canter, expertly led by Sara Nash, where we shared practical insights on implementing semantic enrichment. Throughout my experience, I’ve learned a few simple tricks that significantly improve the readability, discoverability, and meaningfulness of a data estate. Here are the Top 3 that come to mind: 1. Master Metadata, Taxonomy, and Ontology   These are the building blocks of all the data you have — the “fishbone” that gives your data its core meaning. By defining and connecting referential data, you enable organizational units to work toward common goals and execute advanced analytics. It’s both a one-time setup and a continuous effort, as data evolves alongside business needs. So, plan accordingly for this dynamic process. 2. Build Simple Data Products for User-Centric Abstraction   Often, we have limited control over the semantics of source data (the infamous “systems of record”). Despite their continued relevance to the business, these systems can be challenging to decipher. Data products offer a great solution, allowing you to abstract from the source and add meaning tailored to the needs of your users. This makes your data estate far more user-friendly and impactful. Check excellent post by Karthik Ravindran about why Data Products matters. 3. Discover Data Governance for Unstructured Data   Historically referred to as “Data Lifecycle” management, this approach helps enhance the discoverability of unstructured data by focusing on policies, frameworks, archival strategies, and more. Take a closer look at your unused or partially used data — tools like the “Unused Data Framework” that Tiankai Feng and I presented a few weeks ago can help. The goal is to distill what’s meaningful and leave behind what’s not, improving the overall discoverability of your data estate. A big thank you to the organizers — Zach Wahl, Lulit Tesfaye, Benoit Gaussin, and Sara Nash — and kudos to all the presenters, including Malcolm Hawker, Daniel Canter, Tiankai Feng, and Norbert Gergely. Looking forward to the next year – with vectors winning over relational data, the conversation about semantics will received entirely new spin.. ;) #Metadata #SemanticLayer #DataGovernance #InformationManagement #EnterpriseKnowledge #genAI

54

5 Kommentare

Dask DataFrame is fast now - Florian Jetter (Coiled) @ PyData Südwest Live stream of the PyData Südwest meetup https://lnkd.in/eU9G8rvu 20:15 Talk / Q&A 20:45 Talk Lightning Talks Ask questions via Slido: https://bit.ly/SW2405QA 📺 Dask DataFrame is fast now - Florian Jetter (Coiled) Dask is a library for distributed computing with Python that integrates tightly with pandas. Historically, Dask was the easiest choice to use (it's just pandas) but struggled to achieve robust performance (there were many ways to accidentally perform poorly). The re-implementation of the DataFrame API addresses all of the pain points that users ran into. We will look into how Dask is a lot faster now, how it performs on benchmarks that is struggled with in the past and how it compares to other tools like Spark, DuckDB and Polars. ⚡️ Lightning Talks

3

Blogged: Interoperability between lakehouse table formats with Apache XTable (Incubating). Scenario: Dremio -> Apache Hudi ❌ Dremio -> Apache Iceberg ✅ There are 2 analytics teams in an organization. Team A uses Hudi to manage some of their most critical, low-latency data pipelines. Team B focuses on ad hoc analysis, BI, and reporting, using Dremio's compute engine with Iceberg. ❌ Challenge: Team B wants to do analytics using data from the 2 different datasets within the organization. To do so, Team B has to use the dataset generated by Team A (stored as a Hudi table) and combine it with their dataset (the Iceberg table) ✅ Solution: Interoperability between the formats with Apache XTable. Detailed reading link in comments. #dataengineering #softwareengineering

105

4 Kommentare

#DataMasking is not easy. As you can see from several posts from the #SyntheticData world, they assume that anonymizing data is not possible - we Mage Data beg to differ. We have been successfully implementing solutions across global giants, passing penetration tests, getting patents, etc. But we also have fun. Here is a simple example - when Data Masking works, it has to create useful data - useful for the purpose that it is being generated. But it should also be irreversible - so it cannot be used in data theft. We call it "Perfectly Useful, Entirely Useless" (tm) data. Just created a small video to illustrate why old data masking approaches either made the data useless, or left it too easy to reverse. In my example, all identifiers - whether direct, or indirect; whether obvious, or not - were identified and anonymized. Please share your comments! Would love to hear your thoughts, and learn more. #Anonymization #GDPR Shout out to our friends Kalaido.ai - we used Mage Data to anonymize the data, and then Kalaido to generate images - great tool, and we are having tons of fun with it.

49

“After one year of reading and working daily with LLMs, here are the 𝘁𝗼𝗽 𝟭𝟮 𝗯𝗹𝗼𝗴𝘀 on 𝗟𝗟𝗠𝘀 & 𝗥𝗔𝗚 I 𝗿𝗲𝗮𝗱 𝘄𝗲𝗲𝗸𝗹𝘆 and 𝗿𝗲𝗰𝗼𝗺𝗺𝗲𝗻𝗱 ↓”

8

1 Kommentar

Because so many have asked: a great project when it comes to dynamic load balancing in #Proxmox. Hopefully soon also directly integrated, but until then an excellent solution.

15

1 Kommentar