Connor Wade’s Post

DBeaver's Data Editor gives you the freedom to work with document databases the way you prefer. Explore and modify your data in a familiar grid view, or dive into the details with a dedicated JSON view that provides a clear, structured format for easy analysis. Seamlessly switch between these modes to efficiently manage and update your data.

Concat, Join, Merge – Three Ways to Combine Data in Pandas Concat: Adds data either row-wise or column-wise, but it doesn’t look at indexes unless you explicitly tell it to. By default, it adds "blindly," meaning it doesn't care if the indexes align or not. Join: Combines data side by side based on the index. If the indexes don’t match, you’ll get missing values (NaN) in those places. Merge: Combines data based on common values in one or more columns (called keys). Unlike concat or join, which focus on stacking or aligning indexes, merge looks at the actual content of the columns to find matches. Hoping now you can relate each other.

🚀 𝐒𝐐𝐋 𝐑𝐨𝐚𝐝𝐦𝐚𝐩 🚀 𝐋𝐞𝐯𝐞𝐥 1: 🏗️ 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐃𝐚𝐭𝐚 𝐓𝐲𝐩𝐞𝐬 & 𝐎𝐩𝐞𝐫𝐚𝐭𝐨𝐫𝐬: Understand integers, strings, and master SUM, AVG, COUNT. 𝐒𝐄𝐋𝐄𝐂𝐓 & 𝐖𝐇𝐄𝐑𝐄: Extract data using SELECT and WHERE clauses. 𝐉𝐎𝐈𝐍𝐬: Connect tables like a pro with INNER, LEFT, RIGHT, and FULL JOINs. 𝐋𝐞𝐯𝐞𝐥 2: 🧠 𝐈𝐧𝐭𝐞𝐫𝐦𝐞𝐝𝐢𝐚𝐭𝐞 𝐆𝐑𝐎𝐔𝐏 𝐁𝐘 & 𝐇𝐀𝐕𝐈𝐍𝐆: Organize data and uncover trends. 𝐎𝐑𝐃𝐄𝐑 𝐁𝐘 & 𝐋𝐈𝐌𝐈𝐓: Sort and control results for clarity. 𝐒𝐮𝐛𝐪𝐮𝐞𝐫𝐢𝐞𝐬 & 𝐂𝐓𝐄𝐬: Level up with subqueries and Common Table Expressions. 𝐋𝐞𝐯𝐞𝐥 3: 🚀 𝐀𝐝𝐯𝐚𝐧𝐜𝐞𝐝 𝐖𝐢𝐧𝐝𝐨𝐰 𝐅𝐮𝐧𝐜𝐭𝐢𝐨𝐧s: Peer into the future with ROW_NUMBER(), LAG(), and LEAD(). 𝐒𝐞𝐭𝐬 & 𝐅𝐮𝐧𝐜𝐭𝐢𝐨𝐧𝐬: Unleash power with INTERSECT, UNION, and EXCEPT. 𝐏𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧: Tune queries for peak performance with indexes and query plans. #sql

#Spread Data Logger - indicator for MetaTrader 4 https://lnkd.in/e2SJGUxa # Are you getting the spread you've been quoted by your broker? Ever wanted to verify spread data stats from another source or create your own data? Description: Spread Logger consists of two indicators: Spread Logger Write - Creates a CSV ...

Day 4/100 .. I am still working on retrieving the right answer through the code interpreter. One problem I am dealing with is that I need to make it extract column names and use those instead of reading the whole file. Reading the entire file is a bad idea because it can contain thousands of rows. I want to generate the right code based on the question and use that to find the answer using the code interpreter tool. What I am trying to build is a way for business owners to get actionable insights from their data. Feel free to DM me if you have trouble getting insights from your data. Join the waitlist: https://lnkd.in/enG2259H

In 2020, I introduced a new feature to Flookup Data Wrangler for highlighting duplicates by similarity. The idea was to allow users to highlight duplicates based on how similar text entries were to one another. I decided to release the function with two versions: highlight by percentage similarity and highlight by sound similarity. The first version compared every text entry in a column to other text entries in that column and determined, through fuzzy matching, how similar the text entries were. If they were similar beyond a certain threshold, the entry would be marked as a duplicate. The second mode operated similarly but relied on a modified version of the Soundex algorithm. These features were particularly exciting for me as I had not seen anyone else offering this online, even to date. However, due to the nature of the process, if you had 5 unique entries, each having about 3 duplicates, your spreadsheet would be a sea of yellow or teal (the colours I chose for highlighting duplicates by percentage and sound similarity, respectively). How, then, would Flookup users be able to tell which duplicates belonged to which unique entry in the dataset? Enter a feature I call "Trace Duplicate Clusters", launched this year. By using this feature, you can isolate a single unique entry and then highlight other text entries in your list that are its duplicates. To learn how to use this specific function, please click here: https://lnkd.in/eFApVh82 #Data #LeadManagement #DataCleaning #CRMsolutions

◆◆Eager and lazy loading In orm framework . 1- Eager loading - When we work in with large data set .in single query we load all data. that is eager loading example -By using include method we implementing Eager loading 2- Lazy loading - When we required data that time we load only data.mean on demand we load data .that is lazy loading Example - By using virtual keyword in properties we implement Lazy loading 😊😊

📃 Real World Data Prep for LLMs: Why is PDF parsing so difficult? 👩💻 Shuveb Hussain, as the co-founder of Unstract, which addresses structured data extraction problems using LLMs, is well-versed in the various challenges of data extraction. Watch this short tutorial by Shuveb Hussain with Yujian Tang with OSS4AI, focusing on the challenges and solutions in real-world data preparation for LLMs. In this video, watch as Shuveb shares the problems that are prevalent throughout the PDF extraction landscape, and how you can extract structured data from complex documents. Here are some of the most common text extraction challenges: ❌ PDFs with Tables ❌Non-linear Text Flow: PDFs often organize text in columns or around images, confusing extraction tools. ❌Quality of the PDF: Issues like lighting conditions, rotation, skew, and compression levels of the original photo can degrade text extraction quality. ❌Page Orientation: Extracting text from PDFs with both portrait and landscape modes is more complex than uniform page orientations. ❌Handwritten Forms: Not all OCRs can recognize handwritten text. ❌Checkboxes and Radio Buttons: Many text extractors struggle with these elements, though pdf.js does a good job, it’s not always feasible to use third-party services. 👨💻 When faced with these challenges, manual methods often become necessary. In the video, you'll learn: how we take unstructured documents, use a text extraction library/service to extract raw text, and then employ a combination of Pydantic and LangChain to create structured JSON. We will examine different document formats, including tables, PDF forms, and scanned documents, using various libraries and services: ✅ PDF Plumber ✅ Camelot ✅ Tabula ✅ unstructured.io ✅ LlamaParse ✅ Unstract's LLMWhisperer Join us as we delve into these challenges and explore the tools that can make text extraction more efficient and accurate. https://lnkd.in/gnSxfMnE #TextExtraction #pdfextraction #LLMs #datapreparation

One of the causes of complexity that we overlook is names that are too generic. Almost all codebases that I've worked on have had at least one "processData" or "parseData" function that can leave you wondering what they do. In 99% of the cases, we can think of a more specific name. But why do we end up with these in the first place? Because at write time, naming a function "parseData" makes absolute sense. We have all the context about your current task in your head and we need to do some data manipulation. To make sure we can understand it in the future, we need to include the context we take for granted. When you're naming a function or a variable ask yourself this: "Will I understand this without the context of my current task?"

***Float, Double, and Decimal data types*** In the land of SQL, we have these special tools called Float, Double, and Decimal data types, and they're super useful for different tasks. Like Decimal helps us count money exactly, Double is great for super accurate calculations, and Float is awesome for big or small numbers like distances in space. Let's dive in and discover how these tools make our digital world more awesome! Big thanks Codebasics team! #SQL #Databases #TechExplained