Naveen Pasupuleti

Folks of the 80s and 90s - we boast of having witnessed the most transformations that any past generation has seen, don't we? From televisions with knobs to intelligent robots? 'Picture abhi baaki hai mere dost'. I wish that day would come sooner, when going to school to obtain a formal degree is not mandatory for companies to evaluate candidates and provide jobs. The day when internet access on mobile phones, and the phones themselves, is so accessible that even a homeless person with the cheapest phone could learn whatever they wish—whether it's mending a tire puncture, fishing, or creating a website that surpasses the creativity of even what Facebook brought in way back. The day when we, who are privileged would learn what it means to start from the same starting line as everyone else, as opposed to the advantage of beginning the kilometre long race from a mile ahead. With content being created at an astronomical pace on platforms like YouTube, GeeksforGeeks, Medium etc., and AI and other technologies taking the front seat, it's already begun.

8

🕉️ Ancient Wisdom for Modern Peace: Powerful Shlokas from Bhagavad Gita 1️⃣ “Yoga-sthah kuru karmani, Sangam tyaktva dhananjaya.” (2.48) "Perform your duties being steadfast in yoga, renouncing attachment, and remaining even-tempered in success and failure." ✨ Lesson: Focus on your actions, not the outcomes, and embrace equanimity to navigate life’s ups and downs. 2️⃣ “Shreyan swadharmo vigunah, Paradharmat swanushthitat.” (3.35) "It is better to perform one’s own duty, even imperfectly, than to master the duty of another." ✨ Lesson: Focus on your unique purpose and stop comparing yourself to others for a calmer mind. 3️⃣ “Uddhared atmanatmanam, Na atmanam avasadayet.” (6.5) "Elevate yourself through your own mind; the mind can be both a friend and an enemy." ✨ Lesson: Cultivate self-awareness and positivity to transform your mind into an ally. 4️⃣ “Samam kaya-shiro-grivam, Dharayan achalam sthirah.” (6.13) "Keep your body, head, and neck straight, and calm your mind." ✨ Lesson: Practice meditation to achieve mental clarity and reduce stress. 5️⃣ “Yada samharate chayam, Kurmo ’nganiva sarvashah.” (2.58) "Withdraw the senses from their objects, like a tortoise withdraws its limbs." ✨ Lesson: Master self-control and mindfulness to minimize distractions and sharpen focus. 💡 Whether it’s through meditation, mindful actions, or devotion, these teachings are as relevant today as they were centuries ago. Recite, reflect, and integrate these verses into your life to cultivate a calm and resilient mind. #PersonalDevelopment #MentalHealth #Leadership #Mindfulness #BhagavadGita #WisdomWednesday #PersonalGrowth

2

𝗙𝗶𝘅𝗶𝗻𝗴 𝗥𝗔𝗚𝘀 𝘄𝗶𝘁𝗵 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚𝘀: 𝗣𝗮𝗿𝘁 𝟯 Yesterday, I spoke about the steps in a Graph RAG process [https://lnkd.in/dtxZu82K], however, the post didn't include a schematic /flowchart of the process. So, here it is. I have also elborated on the steps as you see below. As I said, an elaborate blog on the subject is yet to come! 𝗘𝘀𝘀𝗲𝗻𝘁𝗶𝗮𝗹 𝘀𝘁𝗲𝗽𝘀 𝗶𝗻 𝗮 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚 𝗽𝗿𝗼𝗰𝗲𝘀𝘀: 1. Source Documents and Text Chunks: This is the same as that of a Standard RAG process 2. Element Instances: Entity Extraction and Relationship Capture. This is essentially building up of Nodes and Edges of the Knowledge Graph. This is the heart of a GraphRAG process. 3. Element Summmaries: This is the summarization of all the nodes and relationships extracted. 4. Graph Communities: Clustring of elements into communities. Clusters of Nodes connected to strong relationships. Hierarchical clustering practiced 5. Community Summaries: Summary of each community - cobering the entire corpus 6. Community Answers: Given a question, each community summary is used to generate a partial response, before all partial responses are again summarized in a final response to the user. #graphs #rags #graphrags #llms Follow Ajay Taneja for insights on: Data Science, Data Engineering, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics My Technical Blogs Library 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq Image Courtesy: https://lnkd.in/ddXW9Dke

5

Types of Indexing in Apache Hudi Index TypeDescriptionUse CasesBloom IndexUtilizes Bloom filters to quickly determine the potential presence of a record in a file group, reducing the search space during upserts and deletes.- High Throughput Upserts/Deletes: Ideal for scenarios with frequent data modifications. - Large Datasets: Efficiently handles big data volumes by minimizing file scans.Global Bloom IndexAn extension of the Bloom Index that maintains Bloom filters across all file groups globally, enhancing lookup performance for massive datasets.- Massive Scale Operations: Suitable for extremely large datasets where maintaining global state significantly improves performance. - Distributed Environments: Enhances consistency and speed across distributed file groups.HBase IndexLeverages HBase to store and manage index data externally, providing low-latency record lookups and high scalability.- Low-Latency Requirements: Perfect for applications needing rapid access to specific records. - External Index Management: When integrating with HBase ecosystems for robust indexing solutions.Simple IndexA non-index-based approach that scans all file groups to locate records. Acts as a fallback when other indexing mechanisms are not enabled or suitable.- Fallback Mechanism: Ensures data operations can proceed even without specialized indexing. - Smaller Datasets: Acceptable for datasets where full scans do not significantly impact performance.In-Memory Hash IndexMaintains an in-memory hash table mapping record keys to their respective file paths, enabling quick lookups without disk I/O.- Fast Lookups: Best for applications requiring rapid access to records with minimal latency. - Memory-Abundant Environments: Suitable when ample memory resources are available to store the hash index.SQL Query Examples Using Apache Hudi Indexes Apache Hudi integrates seamlessly with Spark SQL, allowing you to specify indexing types during table creation or data write operations. Below are examples demonstrating how to leverage different indexing types in your SQL workflows. Example - Creating a Hudi Table with Bloom Index sql Copy code CREATE TABLE hudi_bloom_index_table ( id STRING, name STRING, age INT, city STRING ) USING hudi OPTIONS ( primaryKey = 'id', preCombineField = 'timestamp', hoodie.index.type = 'BLOOM' ); Explanation: hoodie.index.type = 'BLOOM': Specifies the use of Bloom Index for efficient record lookups during upserts and deletes.

6

How Quora Scaled Writes with MySQL ? 1. Initial Setup: They began with a non-sharding setup, scaling reads using a leader-follower configuration + caching. 2. Increasing Write Throughput: They adopted two main approaches: Vertical Sharding: - They moved high-traffic tables to different partition groups, so multiple partitions ran different sets of tables using a master-follower setup. - They used Zookeeper to store information about partitions, their tables, and master and replica instances. - This method worked well for small tables but faced challenges with large tables, joins, and transactional operations. Horizontal Sharding: - This approach involves splitting a logical table into multiple physical tables, known as shards. - They used range-based sharding as they make frequent use of range queries. eg: If a shard of table T holds values >= x and < y, they named it T_shard_x_y. - They used the tool pt-online-schema-change to modify the schema, which requires 2x space as it creates a copy of the original table. Although it is an "online" schema change tool, it initially acquires a short-duration X lock on the entire table to create triggers. This X lock sometimes impacts production traffic. However, if the table has 10 shards, at any given time only about 10% of the table would be locked. This article provides an in-depth explanation and discusses the challenges involved. https://lnkd.in/gWcz26mY.

5

🚨 Tragedy on the Roads: A Call for Action 🚨 The recent accident in Dehradun, where a speeding car driven by young individuals who were under the influence of alcohol tragically collided with a truck, leaving six dead and one in critical condition, is a stark and painful reminder of the dangers we face on our roads. This heart-breaking incident is not just a statistic—it's a call to action. 💔 Lives Lost, Families Broken Each of these accidents leaves behind stories of shattered dreams and grieving families. We must remember that every choice we make on the road has consequences, not just for ourselves, but for everyone around us. 🚦What Can We Do? Stricter Enforcement of traffic laws to ensure compliance. Education and Awareness on the dangers of speeding and driving under the influence of alcohol. Community Involvement to foster responsible behavior and make road safety everyone's priority. Enhanced Penalties for reckless driving and drunk driving to serve as a deterrent. Let's make a collective commitment to safer roads. Every life is precious, and together we can reduce these tragedies. #RoadSafety #DriveResponsibly #EndDrunkDriving #StaySafe

66

#OMSHANTI Urgent Appeal for Justice in the Pune Road Accident Case: Dear Respected Prime Minister Narendra Modi ji,Narendra Modi I am writing to you today to bring to your urgent attention a grave injustice that has occurred in Pune, Maharashtra. Two IT professionals tragically lost their lives in a road accident caused by a high-profile minor, reportedly around 17 years old. It is deeply concerning that this minor was allegedly driving a four-wheeler without a license and under the influence of alcohol. Furthermore, reports suggest the minor received bail within a mere 14 hours on a Sunday, raising serious questions about the handling of the case. This incident is not an isolated one. We have witnessed similar cases in the past where justice seems to falter when it comes to accidents involving high-profile individuals. We, the citizens of India, urge you to take immediate and decisive action to ensure justice is served in this case. This action should include: A thorough investigation into the accident, including the role of: The minor and their guardians who allowed them to drive without a license. The car showroom that potentially sold the vehicle to a minor without proper verification. Any establishments that may have served alcohol to the minor. Stringent punishment for all parties found to be negligent, including the minor and any adults who enabled their actions. A review of existing laws and regulations to prevent such tragedies from happening again. This may involve stricter enforcement of license requirements, harsher penalties for drunk driving, and stricter regulations for car dealerships and establishments serving alcohol. We understand your commitment to a safer and more just India. We believe that taking a strong stance in this case will deter similar incidents, ensuring that such blatant disregard for the law and human life does not go unpunished. We kindly request your intervention and urge you to direct the following authorities to take necessary action: Home Minister of India, Amit Shah ji (https://www.mha.gov.in/en) Government of Maharashtra (https://lnkd.in/gngVm7rX) Home Minister of Maharashtra (https://lnkd.in/gqeVfsTu) Commissioner of Police, Maharashtra (https://punepolice.gov.in/) Commissioner of Police, Pune (https://punepolice.gov.in/) Chief Minister of Maharashtra, Eknath Shinde ji Eknath Shinde (https://lnkd.in/gngVm7rX) Deputy Chief Minister of Maharashtra, Devendra Fadnavis ji Devendra Fadnavis (https://lnkd.in/gngVm7rX) We believe a strong and swift response from your esteemed offices will not only deliver justice in this case but also send a powerful message that such reckless actions will not be tolerated. Thank you for your time and consideration. Sincerely, Purnendu Sekhar Das Purnendu Das On Behalf of Concerned Citizens of India

1

2 Kommentare

Are they Members of deep state who funded propagation of fake narratives by Media & Intellectual Mercenaries and fake promises by Pappu gang of Ghamandi Bandhan? Those who have no time to set right their own future tried to interference in Indian Elections in a big way. Modi had the Majority for 10 years and did nothing to control & punish fake news & narratives nor fake poll promises of Freebies nor controlled the game played in the media by digital platforms such as YouTube, Facebook, WhatsApp etc. Modiji's craving for personal image abroad by ignoring things in Best National interests has lead Karma to show the Mirror to Modiji with a dented image & reputation in India and abroad. Hope Modiji realizes that Wars including political wars on from digitally in the information age. Hope I&B Ministry takes preventive and corrective actions without delay. Modiji's IT cell & cadres should learn from opposition to create their own ecosystem that works together to effectively counter foreign funded narratives of CONgrease ecosystem. #pmoindia #narendramodi #bjp4india

32

𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗚𝗿𝗮𝗽𝗵 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻: 𝗙𝘂𝗻𝗱𝗮𝗺𝗲𝗻𝘁𝗮𝗹𝘀 𝗘𝘅𝗽𝗹𝗮𝗶𝗻𝗲𝗱 Last week, I discussed Knowledge Graph Completion in this post[https://lnkd.in/dR9pGcEV]. As mentioned earlier, Knowledge Graphs can become massive, leading to missing links within the graph. To address this, we predict these missing links using link triples, which consist of (head, relationship, tail). We then create embeddings for each component of the triple: the head, the relationship, and the tail. These embeddings can be formulated in Cartesian space and depend on the type of relationship, such as: a) Symmetric Relations: e.g., "X is the housemate of Y" (and vice versa, "Y is the housemate of X") b) Antisymmetric Relations c) 1-to-N Relations: e.g., "X is the advisor of A, B, C, D,..." However, in Cartesian space, it might not always be possible to test for the existence of the missing link across all types of relationships mentioned above. This is where the beauty of Knowledge Graph Embeddings comes into play. We transform the problem into another space using a transformation matrix "M_r" and then evaluate if the triple exists. There are various methods to test the existence of the triple, depending on whether you're working in Cartesian space or a transformed space. Some of these methods include: 1. TransE 2. TransR 3. DistMult 4. ComplEx And more. The key is understanding how to transform the triple so that the relationship holds. This is the crux of Knowledge Graph Completion. Watch this space for a detailed blog on: 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗚𝗿𝗮𝗽𝗵 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝘀 𝗮𝗻𝗱 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗚𝗿𝗮𝗽𝗵 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻: 𝗣𝗮𝗿𝘁 𝟭𝟰 𝗼𝗳 𝗺𝘆 𝗚𝗿𝗮𝗽𝗵 𝗦𝗲𝗿𝗶𝗲𝘀 𝗼𝗳 𝗕𝗹𝗼𝗴𝘀 #graphs #knowledgegraphs #deeplearning #embeddings Follow Ajay Taneja for insights on: Data Science, Data Engineering, Finite Element Analysis, Non Linear Finite Element Analysis, Fatigue, Fracture Mechanics My Technical Blogs Library 1) https://lnkd.in/ejyWacXn 2) https://lnkd.in/bfrkRjq

7

Converting a list to string, with a delimiter. Implementation of inbuilt mkString() function in SCALA. #SCALA #SCALADEVELOPER #BIGDATADEVELOPER #BIGDATA Karthik K. def main(args: Array[String]): Unit ={ //var a = "Tanuj" //lowerCase(a) var s = "" var l = List(1,2,3,4,5,6,7,8) for( i <- 0 until l.length) { if( i == l.length -1) { s = s.concat(l(i).toString) } else { s = s.concat(l(i).toString) s = s.concat(':'.toString) } } print("The elements of list when converted to string are:-" + s) } } OUTPUT:

2

Batch Processing Using Airflow and Spark For a data engineer, Apache Airflow and Apache Spark are already familiar tools. Apache Airflow is a workflow orchestration tool that is highly useful for data engineers to manage, schedule, and monitor complex data pipelines. The functions of Airflow include: 1. Workflow automation 2. Task scheduling 3. Monitoring and logging 4. Modularity and scalability 5. Integration with the big data ecosystem 6. DAG (Directed Acyclic Graph) Meanwhile, Apache Spark is a distributed data processing platform used to process large volumes of data quickly. The benefits of using Spark include: 1. Fast data processing 2. Support for various types of data processing, such as batch processing, stream processing, machine learning, and graph processing 3. Integration with the Big Data ecosystem like Hadoop, Apache Hive, Cassandra, Kafka 4. Flexible and Supportive API 5. Parallel and distributed processing 6. Support for Big DataInteractive data analysis Let's connect! #Dataengineer #airflow #DAG #Spark

8

Check out my latest Medium blog to get all the Table or views DDL in single statement to save your time. In this blog, I have shared my recent experience where i had to extract the DDL and compare with different environment. We have GET_DDL function to extract DDL, but challenge was to extract 100 tables . So by this below SQL logic i have inserted all DDL definition in Temp table. Feel free to comment or add anything as per your requirement. #snowflake #script #plsql #ddl

19