Developer cut Fable costs by 60% by turning code into images

A project called pxpipe has appeared on GitHub, offering an unconventional way to cut costs when working with language models. The tool converts source code into images, after which the model recognizes the contents using OCR rather than processing the text directly.

According to the project's author, this trick cut the cost of using the Fable model by 60%. The release drew notable interest on Hacker News, where the news gathered 157 points and 56 comments.

How it works

The idea rests on the difference in how text and image tokens are priced: if recognizing code from a picture costs less than processing the same amount of text directly, converting it into images can yield substantial savings when there's a large volume of code.

The implementation details, the limitations of the approach, and the scenarios it applies to are available in the project's repository on GitHub.

Community reaction

The release gathered dozens of responses on Hacker News, which speaks to developers' interest in unconventional ways of optimizing the costs of working with language models. The summary itself does not go into the details of the discussion.

Source: Hacker News