AI skills pipeline – Lessons from South African astronomy

Two decades ago, South Africa had relatively few astronomers with PhDs, most of them white males. Today, it has hundreds, including many who are black and-or female. How did the country go about producing so many and such a diversity of highly qualified individuals comparatively quickly?

And what lessons can be learnt from this experience that can help the country – and others in a similar position – boost its skills base further to take advantage of the Fourth Industrial Revolution (4IR), including rapid advances in artificial intelligence?

University World News discussed these questions with a range of academics, researchers, officials and policymakers at various science bodies, higher education structures and government departments in South Africa.

Big vision

The country’s swift progress with producing astronomy-related skills can be traced back to three mega science projects in its semi-desert Karoo region since the turn of the millennium – the Southern African Large Telescope (SALT), the Square Kilometre Array (SKA), and its precursor, the MeerKAT radio telescope array.

Once it landed these big projects, South Africa had to pull out all the stops to produce enough skilled personnel in a hurry and maximise the return it would get on its sizeable investment.

“We were nowhere ready, especially not to ensure that these facilities would benefit us in the long run and not just the international partners involved,” Professor Patricia Whitelock, a former director of the South African Astronomical Observatory, told University World News.

A pair of parallel initiatives were started to change things around – the National Astrophysics and Space Science Programme in 2003, followed two years later by the Human and Capacity Infrastructure Development (HICD) programme of what would in time become the South African Radio Astronomy Observatory (SARAO). Both are mainly funded by the Department of Science and Innovation (DSI) through the National Research Foundation (NRF).

National Astrophysics and Space Science Programme

The National Astrophysics and Space Science Programme has a node at three of South Africa’s 26 public universities – Cape Town, KwaZulu-Natal and North-West,with participation from seven others as well as from SAAO and SARAO. It has delivered 466 honours and 236 masters degree graduates in astrophysics and space science in the past 20 years, many of whom have gone on to do doctorates.

Professor Saalih Allie, co-director of the University of Cape Town node, said a bridging programme has been a key component of the programme’s success.

“Initially, it was difficult to produce enough black South African graduates. So, we started winter schools in 2008, where we would recruit students from historically disadvantaged institutions. Then we supported them to go on to honours and beyond,” Allie said.

Not all graduates of the National Astrophysics and Space Science Programme stay in astronomy, but this is not seen as a problem.

“Astronomy is one of the best training grounds for the huge new area of data science. That’s where many of our students go,” said Associate Professor Sarah Blyth, Allie’s fellow co-director. “They are very good at programming and at analysing large data sets, which makes them extremely marketable for the fast-growing broader technology sector.”

SARAO

Kim de Boer is the head of SARAO’s human capital development programme, which has been a success story of note, fostering a skilled workforce for radio astronomy in South Africa and contributing to the country’s broader knowledge economy.

Since its inception 19 years ago, the programme has funded more than 1,500 grants for postdoctoral fellows, postgraduate and undergraduate students in science and engineering, and students training to be artisans involved in telescope construction and maintenance.

These grants have helped develop a pool of skilled professionals in radio astronomy, engineering, science, and related fields crucial for operating and maintaining MeerKAT and contributing to the SKA project.

Academics in the field say De Boer has been central to the success of human capital development in radio astronomy in South Africa.

“She’s a force of nature – absolutely remarkable,” says Professor Dirk de Villiers of the faculty of engineering at Stellenbosch University.

He started out as a postdoctoral fellow funded by the South African SKA Project and has held one of five research chairs linked to the SKA since 2018. Each allocated to a different university, these are all funded by the DSI via the NRF as part of the South African Research Chairs Initiative.

All five chairs are in their third and final five-year cycle. De Boer says plans are being formulated to ensure “we don’t lose the expertise we have invested in, and we don’t lose the momentum of capacity development the chairs have created”.

One of the lessons from the astronomy example is that long-term funding stabilises the skills pipeline. De Boer said: “We are at the tipping point of reaping what we have been sowing for years. HCD is long term, because it involves supporting young people through at least 10 years of education, studies and research to become PhD graduates and enter careers in research-academia etcetera.

“We have pulled this off for the most part, but now it is time to start seeing greater levels of transformation, and hopefully working with the Department of Higher Education and Training to get these graduates employed at universities.”

SARAO also funds research groups – currently eight of them, with five having concluded.

De Boer downplays her own role. “Ours is not a special programme in that it’s better. It’s special in that it was strongly purpose driven from the beginning – to build capacity for the SKA,” she said.

National programme

Everyone University World News spoke to agreed that the sharp national focus so characteristic of the SALT, SKA and MeerKAT programmes has been a key contributor to the fact that the country now has built a strong skills and postgraduate pipeline in astronomy.

The question is whether this approach should also be applied more broadly.

Dr Whitfield Green, CEO of the Council on Higher Education – an independent statutory body responsible for researching and monitoring higher education as a basis for policy advice, and for promoting quality and transformation within South Africa’s higher education system – thinks so.

“We need a comprehensive national programme for doctoral education, with a greater degree of coordinated academic support, mentorship and financial support for doctoral students, including bursary funding,” he told University World News.

“All the existing, somewhat disparate initiatives should be pulled under such an umbrella. Because we currently have an uncoordinated approach. Everyone is basically doing their own thing. We need a proper mapping exercise,” he added.

Between them, 26 public universities and 56 accredited private higher education institutions in South Africa offer 2,986 undergraduate and 5,598 postgraduate qualifications. This includes 2,428 masters and 1,096 doctoral degrees. There are 23 public universities and five private institutions that offer doctoral qualifications.

Size and shape, underfunding and targets

According to Green, the country’s postgraduate system is showing encouraging but insufficient signs of progress. “Participation has increased significantly, particularly by previously marginalised groups. So, the pipeline is growing, but there is still too much attrition,” he said.

From 2005 to 2020, postgraduate enrolments have increased by 53%, but as a proportion of all enrolments, they have remained static at 14%. This falls short of the target of 25% set in South Africa’s National Development Plan.

Enrolments by black South Africans have increased by 82%, while the female share of postgraduate enrolments increased from 52% to 57%.

Enrolment in science, engineering and technology fields is up 10%, from 26% of the total to 36%. It is now easily the biggest category, beating business and commerce by 8%. Postgraduate graduation rates have increased – from 25% to 33% overall. But so have dropout rates, across all qualification types, including honours, masters and doctoral degrees.

One of the most serious challenges facing postgraduate studies in South Africa is that it is underfunded.

Pressure on the fiscus means that the state funding going to higher education and research keeps on decreasing in real terms, taking inflation into consideration, Dr Fulufhelo Nelwamondo, CEO of the National Research Foundation, confirmed to University World News.

The NRF has had to cut back on the research funding it provides to universities, including postgraduate bursaries. In terms of its new postgraduate policy, implemented in 2020, it now covers the full cost of studies, but because bursaries are larger, fewer students are funded.

Professor Johann Mouton, director of the DSI-NRF Centre of Excellence for Scientometrics and STI Policy at Stellenbosch University, told University World News that South Africa would likely not achieve its doctoral targets.

The National Development Plan sets a target of 100 doctoral graduates per million of the population per year by 2030. This translates to roughly 5,000 doctoral graduates needing to be produced by South African universities each year by 2030, but the current annual average is only about 3,500.

Presidential PhD Initiative

A desire for the country to pick up the pace could be behind the announcement in December 2023 of a Presidential PhD Initiative funded with an initial investment of R1 billion from the National Skills Fund.

“The aim of the first phase is to expose our country’s brightest young minds to cutting-edge thinking and research by negotiating opportunities at world-leading universities and research centres,” said South African President Cyril Ramaphosa.

The programme is meant to build “critical skills in areas like artificial intelligence research, advanced biotechnology, fuel-cell development, batteries and other storage, and next-generation mining”.

Ramaphosa called on the private sector and international partners to assist in growing the initiative’s fund to R5 billion by 2030. He also “urged the science sector to work collaboratively and ensure synergy between programmes”.

Disjointed approach

Green is positive about the initiative but indicates that it will be important for the skills that are developed abroad to be drawn into a national initiative to build skills at home – otherwise it could be another example of a “disjointed approach in terms of funding, where we have a range of streams”.

For instance, the National Skills Fund is funding the initiative, but they are supposed to fall under the Department of Higher Education and Training. However, according to the Department of Science and Innovation, the NRF is the “ideal implementing agent” because it is the government’s main postgraduate funding channel.

The DHET and the DSI are the two departments under the Minister of Higher Education, Science and Innovation, Dr Blade Nzimande, who also spoke at the first Presidential Science, Technology and Innovation Plenary where the initiative was announced.

It is therefore ironic that universities have seemingly not been consulted about the initiative. A “strategic engagement session” with the universities must still take place, Imraan Patel, who heads up the research, development and support portfolio of the DSI, told University World News.

Patel explained that the idea is for South African PhD students to go abroad for full-time studies or to acquire specific high-level skills. The roll-out of the initiative will not exclude local universities because “it will build on and strengthen what they are currently doing, including their international partnerships”.

Not ‘grand plan’ for AI skills

Asked whether he thinks a national drive for an AI skills pipeline is needed, similar to what was done in astronomy, Patel said the two areas are completely different.

“The astronomy space is very clearly defined, whereas the AI space is currently a lot messier and there is still a lot of hype. So, it would not help to pump in a lot of money to get our postgraduate numbers up, but there is no directionality,” he said. “Things need to mature and evolve. So, there should be no rush to finalise a kind of ‘grand plan’ for postgraduate development.”

Patel said various efforts were underway to shape clear plans on priority areas, including analysis of the current profile of postgraduates, the state of basic sciences, demand for postgraduate skills, and opportunities for long-term productive research careers.

The DSI supports the Foundational Digital Capabilities Research Programme, which got underway about three years ago. It is looking at the basic skills South Africa needs in the 4IR era.

“We cannot afford not to have capabilities in any of the big technology domain areas that’s driving digital change. But there are a lot of things happening globally and South Africa is a relatively small player in the biggest scheme. So, we must choose strategically and carefully where to target public investment,” Patel said.

Another body looking into what to focus on, is the Centre for Artificial Intelligence Research, a multi-institutional platform across nine universities. It is funded primarily by the DSI and virtually hosted and coordinated by South Africa’s Council for Scientific and Industrial Research.

Community of practice

Universities South Africa has identified the postgraduate pipeline as a strategic area and set up a community of practice on postgraduate education and scholarship.

Professor Stephanie Burton, a former vice-principal of research at the University of Pretoria, leads this effort. She also co-authored the Council on Higher Education’s 2022 Doctoral Degrees National Report, based on its National Review of South African Doctoral Qualifications 2020-2022.

She says it is vital for an “enabling environment” to be in place for postgraduate studies to succeed. This includes student support and a vibrant broader academic and research community.

However, inequities in South African higher education mean that some universities do better than others in this regard because they have more resources and have been doing it for longer. There are also inequities in the student body. Some students are less prepared than others for postgraduate studies and need more support.

Also, due to socio-economic pressures, a high percentage of doctoral candidates are studying part-time because they have to support their families and bursaries are not large enough to help them do that.

Another important factor that Burton highlights is limited capacity for postgraduate supervision. The National Development Plan target is to increase the proportion of academics at South African universities who hold doctoral degrees to 75%, but by 2021 this figure had only reached 42.7%.

Impetus

Despite slow progress due to various challenges, Green is cautiously optimistic about South Africa’s postgraduate pipeline. “It’s a complex challenge, but there is a lot of impetus from the ground up. However, we need political will as well as strategic policy direction from the national government,” he said.

He thinks that the Department of Higher Education and Training, as the body overseeing higher education, must take the lead in strengthening South Africa’s postgraduate pipeline. But when it comes to high-level skills for the 4IR, including artificial intelligence, they should do so in tandem with the Department of Science and Innovation.

Case study, not model

Which brings us back to the question posed as the beginning – can South Africa serve as an example of how to quickly develop a pipeline of high-level skills?

“We can be a case study at best, not a model. Because there a gaps and deficiencies in our postgraduate system, and lots must still happen,” says Green.

He makes a good point, because even flaws and shortcomings can offer valuable insights. While success stories are undeniably inspiring, understanding the challenges in a system can be just as informative.