While the recent Paris Summit addressed the pressing issues surrounding AI safety, the underlying challenge stifling AI adoption is the creation and implementation of innovative AI use cases and systems. AI is regarded as a tremendously influential technology that can greatly impact economic competitiveness. A worldwide race is currently in motion to assist economies in reaping the benefits of this technology. Somewhere across the globe, there’s a potential disruptor awaiting to make its move. The recent introduction of Deepseek highlights the global competitive landscape and the critical need for swift implementation.
In a bid to maintain competitiveness within the Indian AI ecosystem, the Indian government is determined to create its own foundational AI model in a relatively short timeframe. Models developed domestically can foster technological autonomy, uphold cultural identity, promote greater transparency regarding data sources, and bolster public trust, making them more effective for deployment.
Foundational models necessitate extensive and varied data sets for optimal learning. These data sets generally encompass text data—large quantities of text extracted from literature, articles, etc.; audio data, such as speech recordings that train models in speech recognition and synthesis; and image and video data, which supplies content for models and assists with image recognition and video analysis. Multimodal data, which integrates text, audio, and visual data facilitates training models that can comprehend and generate content across various formats.
It is vital to recognize that India boasts one of the most diverse populations globally, with over 22 officially recognized languages and a plethora of cultural practices. Coupled with its unique socio-economic framework, incorporating Indian language data within training datasets will enhance the utility of AI models for a wider audience. This incorporation will enable AI systems to grasp and respect cultural subtleties, idioms, and local expressions while minimizing biases stemming from an over-reliance on data originating from other cultures and nations. The absence of foundational models trained on local datasets can have drawbacks; inadequate training data may lead the model to make erroneous assumptions, provide inaccurate answers, and increase the likelihood of hallucinations. For instance, facial recognition systems primarily trained on non-Indian datasets may perform poorly on Indian facial features, resulting in discriminatory outcomes.
As the focus on AI training intensifies within the AI lifecycle, corporations and governments must invest significantly in reevaluating data availability, storage, processing, and usage for AI model development.
To enhance data availability, there are numerous government agencies with immense data reserves. In addition to social media and user-generated content, libraries and archives, governmental records, educational materials, public records in local languages, television shows, and radio programs all serve as potential data sources.
Despite the existence of this data, there remains a limited amount available for foundational model training due to challenges related to data processing and storage. Data is often compartmentalized across governmental departments and other entities, complicating access and integration. Additionally, many datasets lack completeness or machine-readable formats, necessitating further investment in data processing. Furthermore, a significant portion of the data exists in analog formats and requires digitization and reformatting to be effectively utilized in AI training.
The government can play an essential role in addressing these challenges by formulating an AI data strategy tailored for the Indian AI ecosystem. This strategy would prioritize making data storage processes adaptable and suitable for AI models while addressing data preprocessing concerns. Moreover, it could streamline data management practices, encouraging greater data-sharing initiatives from the private sector and promoting the establishment of data exchanges and marketplaces as envisioned by the National Data Management Office (NDMO). The government would also be pivotal in introducing data governance structures and guidelines that specifically target issues related to data security, privacy, ethics, and data sovereignty. These initiatives could serve as a foundational element in assisting private sector organizations in constructing their data architectures. In the near future, such an approach would provide rapid access to data for startups, researchers, and AI developers since most of this information would be controlled and accessible through governmental channels. As significant transformations occur within the Global AI ecosystem, it is paramount for India to develop foundational models rooted in Indian text, audio, video, and multimodal data. This strategy could thus facilitate the creation and rollout of new use-cases within India, promoting the country’s long-term economic and technological prowess.
In conclusion, the government occupies a prime position to tackle these hurdles, and a coordinated, concerted effort by the Government executed with urgency could yield a substantial provision of data for foundational model training.
The author is Rajnish Gupta, Partner, Tax and Economic Policy Group, EY India.
Disclaimer: The opinions expressed are solely those of the author and ETCIO does not necessarily concur with them. ETCIO shall not be accountable for any damage caused to any individual or organization, directly or indirectly.
