Pros & Cons of Using ChatGPT Plus to Convert Legacy Code

Read time: 4 mins

The Version 1 Innovation & AI Labs is a team dedicated to researching and delivering creative solutions to our customers. They experiment with the latest technologies and conduct ‘Proof of Value’ projects to increase customer awareness of innovative tools and technologies. Through these projects, they showcase how Version 1 can deliver a real difference to our customers’ businesses.  

In our latest project, the Innovation & AI Labs worked with SpiderRock, a financial services technology provider in the US, to help convert their legacy code to Python. Not only was this code costly and time-consuming to maintain, but it also held the customer back from reaching their full business potential. 

The challenge posed by this project was the sheer volume of code that needed to be converted. The codebase comprised of 88 Perl files containing an average of 412 lines of code. Manually converting each line of code was expensive and had the potential to introduce errors.  

 

ChatGPT Plus Speeds Up Conversion Time

The team at the Innovation & AI Labs kicked off their research and delivered an initial Proof of Value (PoV). They chose the latest version of ChatGPT Plus as the best candidate to significantly speed up the conversion process and increase overall accuracy.  

First, a baseline was established by manually converting Perl files of varying lengths, and five Perl files were then converted using ChatGPT Plus. The converted files were validated by comparing them with the actual output files. The result was up to 87% less conversion time.  

 

Bar graph - ChatGPT Plus vs manual code conversion time taken
A comparison of code conversion time when done manually VS using ChatGPT Plus

 

The chart above shows conversion time. The manual process takes 500 minutes on average per file, with ChatGPT the average is just 65 minutes. While ChatGPT was effective in reducing the time required for initial conversion, it also has some limitations. 

 

ChatGPT Plus Limitations

 

Flowchart of ChatGPT Plus code conversion
ChatGPT Plus code conversion flowchart

 

As shown in the above process diagram, the model has a token limitation of 8k characters at a time, equal to around 100 lines of code. So, for very large tasks it is not the best approach. ChatGPT Plus also struggles with identifying which parts of generator programs to modify and which to leave intact. Additionally, it may mishandle f-strings, struggle with variable initialization, and leave comments as placeholders for code and variables within functions.

 

Pros and Cons of ChatGPT Plus

Our experiments highlighted that using ChatGPT can result in significant time savings for the initial conversion process. While this looks promising it is not without limitations. There is a restriction on the length of code that can be processed, which can create bugs that have to manually fixed. For larger lines of code, breaking down the files into smaller parts can yield better outcomes when utilizing ChatGPT. Nonetheless, it remains a valuable resource for developers to improve their Perl/Python skills.

Find out more about the Innovation & AI Labs or talk to us today about how we can help you do more with AI.

Back to top ↑

About Version 1 Innovation & AI Labs

Set up in 2018, our Innovation & AI  Labs develop innovation solutions and proof of value for customers to ensure Version 1 remain on the forefront of disruptive technology. With an annual investment into our Innovation & AI Labs, we explore past present and future projects that our dedicated team have been working on.