Domo Arigato Mr. Roboto – Using AI Chatbots for an initial Stock Quality scoring

As I mentioned in my 2025 Q2 Performance review, my central investing tool is now a dynamic watchlist that prioritizes Companies based on Quality, Valuation and Momentum.

I have already discussed my preliminary approach to momentum the last time.

Now how to approach the quality of a company in an efficient way ?

The main issue here is that I will not be able to do a “full stack” analysis for each and every company I come across. But I want to have at least a quick “starting point” which I could use to decide if it makes sense to dig deeper or not.

I defined 10 criteria that give me a first insight on how a company could fit to my “Beuteschama” or not. If I would want to check these criteria manually, I would need to use several sources, such as TIKR, some stock websites, the companies’’s IR website and the annual and quarterly reports. A process that on average takes me at least 60-90 minutes to get to a conclusion.

So I decided to try out LLMs in order to “outsource” this first step screening and scoring.

I call this the “10 Factor Model”. One remark: When I say interesting to me, this means that in principle I am looking for boring, family/management controlled companies that are playing in relatively attractive markets,  growing over the long term and are conservatively financed.

This is attractive for me as it suits my investment style and risk preference. Other investors might have very different criteria. I have not backtested these criteria in any way, I just assume that companies that score well will do well over the mid- to long term.

At this early stage in the investment process, the model also includes momentum, both, stock price and fundamental momentum to get a first glimpse how the company is doing in this regard.

When a company is interesting enough, I’ll try to substantiate the factors through additional analysis in a second step before I then add it to my “Top 100 watchlist” (or not).

The 10 factors (in no particular order)

1. Shareholder structure

I do like companies that have at least one significant (or a group of shareholders, ideally a family) that ideally allow the company to employ and execute a long term strategy. Ideally, Management itself is part of that Group. As a threshold, I use 25% of voting rights.

2. Operating margin

Here I use simplistically a threshold of 10% EBIT margin in the last FY. I am fully aware that certain business models have higher or lower general operating margins (COSTCO). But as I mentioned before: This is just a first step.

3. Return on Equity

12,5% is the hurdle which I consider to be a decent ROE. Of course, this is also a function of leverage, but at this stage I want to keep things simple.

4. Long term growth

Here, I defined a threshold of 6% CAGR for the EPS over the recent 10 years or, if not available, 5 years. Why 6% ? Because this is slightly higher than nominal GDP over the long term. 

5. Net Debt

A net Debt to EBITDA ration of below 1 gives us 1, everything else is 0.

6. Cyclicality

This factor requires more judgement. I either use the general sector or assume if the company has a very low beta that it is not very cyclical. Non-cyclical is good and a score of 1.

7. EPS Momentum

This one checks if in the last available financial reporting period, EPS has gone up. If yes, then 1, otherwise 0. This is a very crude proxy for fundamental momentum

8. Capital allocation quality

This is once again a criteria, where the model needs to judge information. In the prompt I give 7 examples of which at least 5 should be fulfilled to get to a score of 1. I also give the LLM some KO criteria.

9. Market position & Market attractiveness

Here, I boiled it down to market growth, diversification across countries and top 3 position in a market. I had more complex definitions but they didn’t work well.

10. 6 and 12 month price momentum

A quick check to look at stock price momentum. I use simple 6 and 12 month price performance. If both are positive, the score is one.

Final score:

At the end of the exercise, the LLM should then add the individual scores with a max of 10. I also ask for some KPIs (Market cap, P/E etc.) , information on management, the latest news, a stock chart and a strengths and weaknesses summary. 

The current version of the prompt is embedded here:

https://drive.google.com/file/d/1ZMIf7Nh__6frI8nsyWtcTZOaZfaxO_RF/view?usp=sharing

I would be more than happy to receive suggestions for improvements and exchange “prompt secrets” 😉

General impression & Comparison of different AI models. 

Some remarks upfront: My goal is here to get a quick overview without requiring too much time and effort (no uploads etc.). For this exercise I used ChatGPT 5 Pro, Gemini 2.5 pro, Claude Pro and Mistral & Perplexity with the free versions.

Here are some observations:

1. results for the same prompt can change significantly from one day to another or even one chat to another
2. Within a chat, results tend to get worse when you prompt a few times. It is better to create a new chat every time.
3. Also the output formats can change from prompt to prompt for no particular reason.
4. For more obscure companies, the variation in the results is much higher. One of the examples that I have in the appendix is eurokai which received scores between 5 and 9 out of 10.
5. Sometimes, the models are not able to correctly add up the scores per category. So you get 6 times a score of 1 and the total score is 5.
6. Despite winning the Math-Olympics, calculating a 10 year CAGR from two EPS values seems to be really difficult for all the models.
7. The models use very different sources for market data with a very wide variety of quality
8. The “deep research” modes do not always produce better results. Sometimes it gets worse. For my purpose, the deep research modes take too long in any case.
9. Only Mistral was able to embed stock price charts
10. Often, the results are getting better if you ask the model to just do it again. Which is quite annoying
11. You can also get different results if you prompt the same thing in two different languages (translation provided by ChatGPT for instance)
12. It makes most sense to start with companies you already know well in order to judge where each LLM has strengths and weaknesses.
13. If you make the criteria too complicated, the results often get worse
14. I tried to ask the LLMs to define criteria themselves, but they often didn’t work all that well

At the end of the day, relying on one model, especially for smaller companies is quite dangerous, you need to look at at least at two of them. It is always worth to aks 

Overall, I have to say that my favorite LLM is still ChatGPT, although briefly, ChatGPT5 was really bad and sloppy. This is followed by Gemini and Perplexity, which is very fast.

Claude is fun to use but not very accurate. LeChat from Mistral is still Ok for a free tool. I would never use Grok for obvious reasons.

Summary:

Overall, I am quite impressed how these LLMs work and improve. I had been involved in looking at Chatbot startups 6-7 years ago and the improvement through LLMs is simply breathtaking. Even in the weeks that I have been iterating on this tasks, the models continuously improved, which I find remarkable.

My “live data Query & interpretation” task is clearly not the ideal use case, but it saves time and effort on my side compared to manual work.

As I just use this as a starting point for further research, I can live with the randomness in the results.

To be continued….. 

Bonus song: Styx – Mr. Roboto

Appendix: Examples

Example 1: EVS Broadcast

ChatGPT 5 with the “think harder” prompt to avoid the sloppy quick and dirty result. It took the longes with around 5 minutes.

It came up with an 8 out of 10 score. It actually got the 10 year EPS growth wrong but decided to use the much better 5 year rate which is something I would do as well.

I like the detailed analysis of the Capital allocation chapter. The market segment was so so…

Perplexity came to 8 out of 10. Funnily enough, my blog was referenced as a source in an earlier query, but Perplexity seems to use different sources at different periods of time for the same prompt. It got however the EPS growth wrong. EPS per share has increased by ~6% per year over 10 years and much more over the past 5 years.

Perplexity was really fast.

Google Gemini standard:

Gemini 2.5. came up with 8/10 points. In the market attractiveness section it made one mistake: it considered the EMEA region as one country and therefore did not give a point which is obviously wrong. Other than that the results were Ok.

Claude also got 10 year EPS growth wrong, but still gave it a 1. Otherwise the results were quite decent , too. I liked the output format a lot.

Mistral got to a 6/10. It made a few mistakes, such as the 10 year EPS growth rate, capital allocation policy and the share price performance. Interestingly, it is the only Chatbot that is able to embed share price charts.

Example 2: Eurokai

Eurkai is clearly a little bit harder than EVS, as there is less frequent information, less analyst coverage etc. It is also a more complicated company (company structure, Pref shares et.). Eurokai’s 10 year EPS growth is around 6,8%.

ChatGPT scored 6/10.

It got EPS growth wrong. With the cyclical sensitivity, it gave a 0 despite acknowledging the very low beta. Capital allocation policy could indeed be considered a 0. Market attractiveness is something I would not support as on a look through basis, more than 50% of activity is in one country.

Perplexity came up with 9/10 points. It got EPS growth wrong and was a little optimistic on the capital allocation criteria. It also dodn’t know that Tom Eckelmann is now CEO.But overall the results are very decent.

Gemini 2.5. flash standard this time returned 6/10 points, and had obviously some issued retrieving more recent information. I just opened a new chat after looking into EVS Broadcast and the performance seems to have instantly degraded and contained a lot of mistakes

Gemini in the Deep Research mode got 7/10. Interestingly it tried to calculate the 10 year EPS growth from historic EPS numbers, but ended up calculating only a 9 year CAGR. On the other hand it correctly found out, that from a market attractiveness, Eurokai is relying too much on Germany (40%).

Claude standard came out with 5/10. It made lots of mistakes, especially with ROE and EBIT margin.

On the other hand, Claude dug out some good stuff on compensation and the bios of the Eckelmanns.

Claude in  Research mode also came out with 5/10, but did an error in adding up the factors. It should have been a 6/10.This is something one sees in other models as well.

It came up with some strange stuff, like a non-existing share dilution.

Le Chat from Mistral scored 7/10. It also got the 10 year EPS growth wrong, but also the dominating shareholder percentage. Interestingly it was the only model that embeds stock charts.

10-Faktoren-Analyse: Eurokai KGaA