About
My passion is to drive transformative change through data intelligence and innovation…
Articles by Rajdeep
Contributions
Activity
-
🚀 Revolutionizing Smart Manufacturing with SMARTMAINT IQ! 🚀 Smart Factories are no longer a novelty, but Neudesic, an IBM Company's SMARTMAINT IQ…
🚀 Revolutionizing Smart Manufacturing with SMARTMAINT IQ! 🚀 Smart Factories are no longer a novelty, but Neudesic, an IBM Company's SMARTMAINT IQ…
Shared by Rajdeep Biswas
Experience
Education
Licenses & Certifications
Volunteer Experience
-
Subject Matter Expert
Bureau of Indian Standards
- Present 3 months
Science and Technology
LITD 30 : WG02-DATA Working Group: https://www.services.bis.gov.in/php/BIS_2.0/bisconnect/dgdashboard/committee_sso/composition/41/5
Publications
-
Just-in-Time Inventory Management Optimization Framework Using Cloud Design Patterns for Manufacturing
The AI Journal
Efficiency and responsiveness toward changes in market demand mark the manufacturing industry. With work developing over time, JIT inventory management will enable manufacturers to hold less inventory overstock, produce products using resources in the best possible manner, and further increase operational agility. However, today, in this complex and volatile world market, the implementation of JIT is far more difficult than ever. From fluctuating demands to infringements of supply chains, these…
Efficiency and responsiveness toward changes in market demand mark the manufacturing industry. With work developing over time, JIT inventory management will enable manufacturers to hold less inventory overstock, produce products using resources in the best possible manner, and further increase operational agility. However, today, in this complex and volatile world market, the implementation of JIT is far more difficult than ever. From fluctuating demands to infringements of supply chains, these issues cannot be solved using the traditional inventory systems anymore.
That is where cloud-based design patterns come in. Scalable cloud infrastructure, real-time data analytics, and automation together can enable the creation of a flexible dynamic framework for JIT by manufacturers. In this article, we look at the main cloud design patterns that can help optimize JIT inventory management within manufacturing. -
Efficiency and Benchmarking Using DEA
Springer Nature
This paper presents a novel framework that combines Data Envelopment Analysis (DEA) and Tessellation methods to determine a composite index for comparing the efficiency of schools based on demographic and operational data. Leveraging DEA's ability to compare performance with multiple inputs and outputs using a non-parametric linear programming method, this study utilized a Voronoi tessellation approach to define each school's catchment area and derive its student target population (Serviceable…
This paper presents a novel framework that combines Data Envelopment Analysis (DEA) and Tessellation methods to determine a composite index for comparing the efficiency of schools based on demographic and operational data. Leveraging DEA's ability to compare performance with multiple inputs and outputs using a non-parametric linear programming method, this study utilized a Voronoi tessellation approach to define each school's catchment area and derive its student target population (Serviceable Addressable Market.
Other authorsSee publication -
Transformation of Industrial Solutions: Advisory to Autonomous Systems
Coruzant
The manufacturing industry presently undergoes a radical transformation impelled by the advance of automation, AI, and machine learning. There is increased integration of autonomous systems within manufacturing as the business tries to enhance efficiency, safety, and productivity. Understanding levels of autonomy within such systems is critical as leaders and technology stakeholders shape the future of manufacturing.
In this article, we will also demystify the capabilities of autonomous…The manufacturing industry presently undergoes a radical transformation impelled by the advance of automation, AI, and machine learning. There is increased integration of autonomous systems within manufacturing as the business tries to enhance efficiency, safety, and productivity. Understanding levels of autonomy within such systems is critical as leaders and technology stakeholders shape the future of manufacturing.
In this article, we will also demystify the capabilities of autonomous systems in manufacturing by breaking them down into four key levels: None, Advisory, Assistive, and Autonomous. Each one describes a different degree of intelligence and interaction for machines and humans and explains how manufacturers can take advantage of these advanced systems to drive efficiency into operations, make better decisions, and thereby create smarter, more responsive factories. -
Why Implementing Data Mesh Architecture should be a Crucial Consideration for Manufacturing Organizations
beforeitsnews
The manufacturing industry is well underway with its rapid transformation driven by technologies, data, and automation. In modern manufacturing organizations, the centralized approach in data architecture is becoming increasingly less effective due to the need to manage rich and complex data ecosystems. Data Mesh architecture grants an approach that is decentralized, scalable, and flexible, bringing more efficiency in the management of data.
In this article, we explain in detail the role…The manufacturing industry is well underway with its rapid transformation driven by technologies, data, and automation. In modern manufacturing organizations, the centralized approach in data architecture is becoming increasingly less effective due to the need to manage rich and complex data ecosystems. Data Mesh architecture grants an approach that is decentralized, scalable, and flexible, bringing more efficiency in the management of data.
In this article, we explain in detail the role of the Data Mesh in manufacturing, its value, how it enables long-term success, and why the importance of a Data Mesh heightens when such an organization grows through acquisition. -
Computer Vision on Light Edge
Bellevue University
End to end Computer Vision on Light Edge
Projects
-
A TALE OF THREE CITIES
-
Time series forecasting, anomaly detection, descriptive analytics, visualization, clustering
on the safety (311) dataset published by Azure Open Datasets for Chicago, Boston and New York City using SparkR, SparkSQL, Azure Databricks, visualization using ggplot2 and leaflet.
This work can be used as a workshop, reference material and self-learning for the following concepts, technologies, and platforms:
Data Engineering using SparkR, R ecosystem
Data visualization and descriptive…Time series forecasting, anomaly detection, descriptive analytics, visualization, clustering
on the safety (311) dataset published by Azure Open Datasets for Chicago, Boston and New York City using SparkR, SparkSQL, Azure Databricks, visualization using ggplot2 and leaflet.
This work can be used as a workshop, reference material and self-learning for the following concepts, technologies, and platforms:
Data Engineering using SparkR, R ecosystem
Data visualization and descriptive analytics
Time Series forecasting
Anomaly detection
This research work got published as an official Microsoft SparkR sample.
https://github.com/microsoft/A-TALE-OF-THREE-CITIES
I think the best way to gauge the work would be from the browsing the sections in the wiki: https://github.com/microsoft/A-TALE-OF-THREE-CITIES/wiki
My favorite part of the visualization is a fully explorable geoplot created using leaflet with ESRI plugin
https://blob9cognitive.z21.web.core.windows.net/rwidgets_safety_events_mapexploration.html
-
Parsing extremely large compressed XML file using spark streaming
-
Production data from New Mexico is published each month in a zipped, 36 TB XML file. It contains data for 55,000 wells over the past 30 years. The file grows in size by 300 GB per month. This repository uses spark code to process the file.
Design Considerations
We have a more than 38 trillion rows from 1970s to 2020.
This file had some unique challenges which I wanted to share:
The XML file did not have any unix or windows style line endings. For a processing engine like spark for…Production data from New Mexico is published each month in a zipped, 36 TB XML file. It contains data for 55,000 wells over the past 30 years. The file grows in size by 300 GB per month. This repository uses spark code to process the file.
Design Considerations
We have a more than 38 trillion rows from 1970s to 2020.
This file had some unique challenges which I wanted to share:
The XML file did not have any unix or windows style line endings. For a processing engine like spark for example it means we are asking to process a string which is of size 36 TB.
Pertaining to point 1 above, one obvious solution can be using a custom line ending. Well it turns out to parse a custom line ending the spark engine will try to read at least one line. In our case it is of size 36TB and it is going to fail since it exceeds the max line size.
Obviously based on point 1 and 2 any kind batch processing will fail (I tried with machines having 512GB RAM, but it did not work)
Hence, I had to use streaming. Now, stream processing uses bytecode to read the files hence multiple times I had to decode the file to get rid of special encoding characters. For example, the streaming will automatically decode it as UTF16 to read the large chunks. I also made it a point to not use custom readers until all options are exhausted. Hence, I had to switch between streaming, batching, spark sql, data frames and make the best use of available methods..
Given below is the high level flow:
Create Azure databricks mounts
Download and unzip the large file to Azure Blob Storage
Use streaming to read the file in small chunks and parse the new lines
Write the streaming results
Run batch processing (transformations/decoding etc) on the results of the streaming output files
Write it to the final destination. -
AI_Enabled_Image_Bucketization
-
Harvest exhaust and AI enabled data on images, pdf and eventually videos for an all out knowledge mining. Additionally build out reusable modules, easy to use code examples and store the search index in a low latency store/db.
-
FindMyImage
-
Use AI and exhaust data mining to arrange, tag, categorize, generate caption and search through all your images in a flash.
Honors & Awards
-
Innovation Leader
Neudesic
Selected for building and evangelizing new professional grade IPs and collaterals using Azure Open AI and Copilot.
-
Most Valuable Contributor
Microsoft
Selected for most innovative idea in Data and AI product portfolio and executing on it in a record time.
-
Hero - Connected Customer Experience
Microsoft
1) Subject matter expertise and tech intensity – Big Data and Analytics Champion
2) Leveraged learnings and patterns from other architects to accelerate consumption
3) Led with empathy to connect and win customer confidence to move the project forward -
Compete Expert
Microsoft
Selected for a large Azure data analytics platform win and for contributing assets to the Data & AI community
-
Most Promising New Entrant
Cognizant
-
Power Programmer
Infosys
-
Employee Champion
Infosys
Organizations
-
Bureau of Indian Standards
Expert Member
- Presenthttps://www.services.bis.gov.in/php/BIS_2.0/bisconnect/dgdashboard/committee_sso/composition/41/5
-
Forbes Technology Council
Executive Member
- Presenthttps://councils.forbes.com/profile/Rajdeep-Biswas-Global-Vice-President-Industry-Solutions-Neudesic-an-IBM-Company/2dcd30ec-9159-420d-a772-ca99d2418824
-
The Institute of Industrial & Systems Engineers (IISE)
Member
- PresentFounded in 1948, The Institute of Industrial and Systems Engineers, formerly the Institute of Industrial Engineers, is a professional society dedicated solely to the support of the industrial engineering profession and individuals involved with improving quality and productivity. Industrial and systems engineering is concerned with the design, improvement and installation of integrated systems of people, materials, information, equipment and energy. It draws upon specialized knowledge and skill…
Founded in 1948, The Institute of Industrial and Systems Engineers, formerly the Institute of Industrial Engineers, is a professional society dedicated solely to the support of the industrial engineering profession and individuals involved with improving quality and productivity. Industrial and systems engineering is concerned with the design, improvement and installation of integrated systems of people, materials, information, equipment and energy. It draws upon specialized knowledge and skill in the mathematical, physical, and social sciences together with the principles and methods of engineering analysis and design, to specify, predict, and evaluate the results to be obtained from such systems.
-
The International Society of Automation (ISA)
Senior Member
- PresentFounded in 1945, The International Society of Automation (ISA) is a non-profit professional association of engineers, technicians, and management engaged in industrial automation. As the globally trusted provider of foundational standards-based technical resources for the profession, ISA strives to build a better world through automation.
-
American Association for the Advancement of Science
Member
- PresentThe formation of American Association for the Advancement of Science (AAAS) in 1848 marked the emergence of a national scientific community in the United States. While science was part of the American scene from the nation's early days, its practitioners remained few in number and scattered geographically and among disciplines. AAAS was the first permanent organization formed to promote the development of science and engineering at the national level and to represent the interests of all its…
The formation of American Association for the Advancement of Science (AAAS) in 1848 marked the emergence of a national scientific community in the United States. While science was part of the American scene from the nation's early days, its practitioners remained few in number and scattered geographically and among disciplines. AAAS was the first permanent organization formed to promote the development of science and engineering at the national level and to represent the interests of all its disciplines. AAAS MISSION: Advance science, engineering, and innovation throughout the world for the benefit of all. Advancing Science, Serving Society. AAAS VISION STATEMENT: A boldly inclusive, mobilized, and global scientific community that ignites, enables, and celebrates scientific excellence and science-informed decisions and actions.
-
Association for Manufacturing Excellence
Member
- PresentThe Association for Manufacturing Excellence (AME) is the premier organization for the exchange of knowledge in enterprise excellence. Members come together to explore lean thinking and other enterprise improvement methods, exchange best practices and network in order to advance their careers and improve the competitiveness and overall value of their organizations.
-
SME
Member
- PresentSME has supported manufacturing for 90 years, from the founding as the American Society of Tool Engineers and evolving to the Society of Manufacturing Engineers in 1969 to becoming SME in 2013. Working closely with manufacturing professionals, companies, educators, schools and communities, SME share knowledge and resources that generate solutions to manufacturing industry challenges.
-
Azure User Group - Houston
-
This group is for anyone interested in a discussion with others local to the Houston area around any aspect of Microsoft Azure (https://azure.microsoft.com/en-us/) and it's rapidly-growing, surrounding (non-Microsoft) ecosystem.
-
Digital Twin - Microsoft IoT & AI - Texas
Co-organizer
This group sponsored by Microsoft is a group for anyone interested in the new power couple IoT & AI. Our goal is to learn by exploring solutions leading to the rise of a customer’s market dominance fueled through their strategic investment in AI and IoT. Source first-hand insights on what enables innovative organizations to push past a pilot and embrace bleeding edge patterns as a differentiator to build future success. Work through advance workloads driven by Azure IoT Ecosytem. We…
This group sponsored by Microsoft is a group for anyone interested in the new power couple IoT & AI. Our goal is to learn by exploring solutions leading to the rise of a customer’s market dominance fueled through their strategic investment in AI and IoT. Source first-hand insights on what enables innovative organizations to push past a pilot and embrace bleeding edge patterns as a differentiator to build future success. Work through advance workloads driven by Azure IoT Ecosytem. We continuosely learn and explore the latest Microsoft IoT technology solutions such as IoT Central, Azure Sphere, IoT Hub, IoT Edge, Windows 10 IoT, and our intelligent edge partner solutions enabling scenarios which pull through increase Revenue for customers. With many customers starting their IoT journey now, we will learn how to combine solutions for the intelligent Edge and the Intelligent cloud with a quick time to realization of your projects in a real hands-on learner mode.
-
Houston Hadoop, Spark, and Machine Learning Meetup group
Co-Organizer and Host
Back in 2012, we said: it is time to have a Hadoop group in Houston, to learn about Big Data and related technologies. Since then we grew from 6 to over 1,500 people, changed our name a few times, and had over 70 meetings, month after month. Here is our story in blog form (http://shmsoft.blogspot.com/search/label/Houston%20Hadoop%20Meetup). This group for anybody interested in Hadoop/Spark/NoSQL/Search/Machine Learning - software developers, managers, entrepreneurs, recruiters.
-
International LEAP Network
Executive Member
Recommendations received
11 people have recommended Rajdeep
Join now to viewOther similar profiles
Explore collaborative articles
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
Explore MoreOthers named Rajdeep Biswas
-
Rajdeep Biswas
🎥 Founder at "Unheard Katha with Rajdeep" 🚀 12 million views in 6 months ⏳️ 40k hours watch time 📈 Helping Podcasters grow on Youtube and Social Media no one manages me (be aware of scams)
-
Rajdeep Biswas
-
Rajdeep Biswas
-
Rajdeep Biswas
BI | Analytics | Data Visualization | Cloud | Agile | Ex-Deloitte
135 others named Rajdeep Biswas are on LinkedIn
See others named Rajdeep Biswas