Topic label and words not representative of tweets

[j-jsmith]

Hi everyone,

Could someone tell me what they would do?

I have used BERTopic on Tweets posted by French politicians. Topic 5 is supposedly about this mini-series on French politicians going through Europe on a train. It's actually a Tweet that has been retweeted 5 times. The rest of the Tweets mention the war in Ukraine. See the images below.

What woud you do? How can I separate these 5 Tweets from the rest?

Thanks in advance!

[MaartenGr]

Apologies for the late reply! It might be that you have set min_topic_size too large or that you have merged topics through nr_topics="auto". Could you share the full code that you have been using?

[j-jsmith]

No problem! Thank you for answering at all!

This is the code that I have been using:

with open('french_stopwords.txt') as f: #Extensive list of French stopwords
stopwords_fr = f.read().splitlines()

embedding_model = SentenceTransformer("paraphrase-multilingual-mpnet-base-v2")

umap_model = UMAP(n_neighbors=15, n_components=10, min_dist=0.0, metric='cosine', random_state=42)

hdbscan_model = HDBSCAN(min_cluster_size=10, metric='euclidean', cluster_selection_method='eom', prediction_data=True)

vectorizer_model = CountVectorizer(stop_words= stopwords_fr, ngram_range=(1, 2))

ctfidf_model = ClassTfidfTransformer()

representation_model = KeyBERTInspired()

topic_model = BERTopic(
embedding_model=embedding_model, # Step 1 - Extract embeddings
umap_model=umap_model, # Step 2 - Reduce dimensionality
hdbscan_model=hdbscan_model, # Step 3 - Cluster reduced embeddings
vectorizer_model=vectorizer_model, # Step 4 - Tokenize topics
ctfidf_model=ctfidf_model, # Step 5 - Extract topic words
representation_model=representation_model,
verbose=True)

An image of my code:

[j-jsmith]

I also tried it this way:
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine', random_state=42)
hdbscan_model = HDBSCAN(min_cluster_size=10, metric='euclidean', cluster_selection_method='eom', prediction_data=True)

But it didn't change...

[MaartenGr]

What woud you do? How can I separate these 5 Tweets from the rest?

One thing that you can do is lowering min_cluster_size to get more micro-clusters. That way, you get more specialized clusters that you can reduce afterwards if necessary.

I also tried it this way:

It seems that the only thing you changed was n_components which does not relate to the size of the clusters directly. Instead, I would advise lowering min_cluster_size which directly impacts the topic assignment.

[j-jsmith]

That did the job, thanks so much!!!!

[MaartenGr]

That's great! Glad to hear that it worked out. If you have any other questions, feel free to reach out.