Skip to content
Notifications
Clear all

Check out my script that cleans up HTML before sending to Speechify for cleaner audio.

22 Posts
20 Users
0 Reactions
197 Views
(@ide_tinkerer)
Reputable Member
Joined: 7 months ago
Posts: 338
Topic starter   [#25189]

Hey folks, been deep in the trenches trying to get Speechify to play nice with some gnarly technical documentation and blog posts I've been converting. I kept noticing the audio would have these weird pauses or mispronunciations, especially around code snippets, navigation elements, and those pesky inline ads.

After some digging, I realized the raw HTML I was feeding it was the culprit. Speechify's parsing is decent for clean articles, but developer-focused content? Not so much. It was reading out `div` class names, script tags, and even "sidebar" text! So, I did what any IDE plugin enthusiast would do—I built a preprocessing script.

The core idea is simple: strip out everything that isn't the core article text before it even hits Speechify. I use Node.js and `jsdom` to parse, but you could adapt the logic to Python with BeautifulSoup or whatever your stack is. The key is being surgical.

Here's the heart of my cleaner script. It targets common noise:

```javascript
const cleanHTMLForSpeechify = (htmlString) => {
// Load the HTML into a virtual DOM
const dom = new JSDOM(htmlString);
const document = dom.window.document;

// Elements to REMOVE entirely
const selectorsToRemove = [
'script', 'style', 'nav', 'header', 'footer',
'.sidebar', '.ad', '.advertisement', '[class*="ad"]',
'.social-share', '.comments', '.related-posts',
'noscript', 'iframe', 'form', 'button'
];
selectorsToRemove.forEach(selector => {
document.querySelectorAll(selector).forEach(el => el.remove());
});

// Elements to UNWRAP - keep the text, ditch the tag
// Useful for inline code or links that disrupt flow
const selectorsToUnwrap = ['a', 'code', 'pre'];
selectorsToUnwrap.forEach(selector => {
document.querySelectorAll(selector).forEach(el => {
const parent = el.parentNode;
while (el.firstChild) {
parent.insertBefore(el.firstChild, el);
}
parent.removeChild(el);
});
});

// Optional: Isolate the main content if a specific container exists
const mainContent = document.querySelector('article, .post-content, main') || document.body;

// Get the final, clean text
let cleanText = mainContent.textContent;

// Clean up excessive whitespace and line breaks
cleanText = cleanText.replace(/s+/g, ' ').trim();

return cleanText;
};
```

Why go through this trouble?
* **Removes auditory clutter:** No more hearing "sidebar sidebar sidebar" or "click here button."
* **Improves pacing:** Gets rid of those jarring pauses where hidden elements were parsed.
* **Handles code better:** Unwrapping `code` tags often works better than leaving them in, as Speechify can stutter on backticks or markup.

I run this as part of a little automation pipeline before I send the text to Speechify's API. The difference in listening experience is night and day, especially for dense, code-heavy articles. It feels less like a robot reading a webpage and more like a coherent audiobook chapter.

Has anyone else tried similar preprocessing? I'm curious if you've found specific HTML patterns or site structures that particularly trip up Speechify's TTS engine. Also, wondering if a browser extension that does this on-the-fly before the Speechify extension grabs the content would be useful.


editor is my home


   
Quote
(@henryg)
Honorable Member
Joined: 4 months ago
Posts: 420
 

So you're pre-processing HTML to compensate for the parser in a paid service. Why not just use a local text-to-speech engine with a proper extraction library to begin with? You're adding a Node step to fix a vendor's problem.


Your vendor is not your friend.


   
ReplyQuote
(@eval_rookie_42)
Honorable Member
Joined: 8 months ago
Posts: 445
 

So you're pre-processing HTML to compensate for the parser in a paid service. I get the other comment's point about fixing a vendor issue, but sometimes the workflow is just locked in, right? You're already using the tool.

My question is about reliability. How consistent is your selector list across different sites? I'm worried I'd spend more time tweaking the rules for each new blog than I save. Do you have a fallback for when the main article text isn't in a standard tag?



   
ReplyQuote
(@gardener42)
Reputable Member
Joined: 4 months ago
Posts: 391
 

Your concern about selector consistency is the key challenge with this approach. I've found that a multi-pass heuristic tends to be more reliable than a static list. My own script does this:

1. Primary extraction using common article selectors (article, .post-content, etc.)
2. A fallback that uses a combination of metrics: the largest text block with proper paragraph density, excluding areas high in link density or script-like class names.
3. A final cleanup pass that removes any remaining elements with `code`, `nav`, or `ad` in their class or ID.

It still isn't perfect for every single site, but it gets you 90% there without manual tuning for most modern blogs and docs. The real time-saver is when you cache the successful selector per domain after the first successful run.



   
ReplyQuote
(@eval_rookie_42)
Honorable Member
Joined: 8 months ago
Posts: 445
 

That approach with the multi-pass extraction makes sense. I've been looking at Readability.js for similar tasks. Does your script handle cases where the main content is split across multiple non-standard containers, like a tutorial with step-by-step divs?



   
ReplyQuote
(@data_pipeline_newbie)
Reputable Member
Joined: 6 months ago
Posts: 292
 

Oh wow, this is really cool and super relevant to a problem I'm hitting right now! I'm trying to automate some documentation audio myself, and the parser reading out sidebar stuff is exactly the kind of headache I'm getting. I'm a bit new to this, so your example with jsdom is super helpful.

I do have a question though, because I'm a bit overwhelmed by selectors. For the list of elements to remove, are you targeting specific CSS classes from certain sites, or is it more general? Like, do you have a big list of common class names for ads and navs, or are you just removing whole element types like 'nav' and 'aside'? I'm worried I'd miss something sneaky.



   
ReplyQuote
(@cassie2)
Honorable Member
Joined: 4 months ago
Posts: 546
 

Totally get that selector overwhelm! When I started, I tried the big list of common class names (like 'advertisement', 'sidebar', 'widget') but it felt like whack-a-mole. New sites kept popping up with weird class names.

My approach now is a combo: start by removing entire semantic element types (`nav`, `aside`, `footer`) which catches a lot. Then, I run a second pass that looks for any element where the class or id contains certain substrings. My shortlist is: `ad`, `banner`, `modal`, `cookie`, `popup`, `menu`, `script`, `code`. That catches most of the sneaky stuff without needing a huge dictionary.

The real pro-tip? After you clean, do a quick visual check of the remaining text in your console. You'll quickly spot patterns for the next site. What kind of docs are you working with?



   
ReplyQuote
(@cipher_blue)
Honorable Member
Joined: 7 months ago
Posts: 506
 

That substring trick is clever, but I worry it's a bit brittle for professional docs. What about legitimate content blocks with classes like `modal-dialog` for an interactive tutorial step or `banner-alert` for a security warning? You'd strip out crucial text.

A more surgical approach is to also check the parent chain. If an element with `modal` in the class is nested inside something with `article` or `main`, maybe keep it. If it's a direct child of `body`, nuke it. Adds complexity, but reduces false positives.

What do you do for developer docs where `code` in a class might actually be a wrapper for a sample you *do* want read aloud?



   
ReplyQuote
(@averyt)
Reputable Member
Joined: 4 months ago
Posts: 274
 

Yeah, that parent chain check is a smart refinement I've started adding too! It does add complexity, but for my docs workflow it's worth it to keep those important alerts and interactive steps.

For code samples, I take a different tack. Instead of stripping them, I replace the entire `` or `

` block with a simple placeholder like "[code block]". That way the audio flow isn't broken by gibberish, but the listener still knows something was there. Sometimes I'll even extract the code language from the class and say "[Python code block]" for a bit more context.


Automate all the things


   
ReplyQuote
(@devops_rookie_2025)
Prominent Member
Joined: 6 months ago
Posts: 467
 

That placeholder trick for code blocks is brilliant! I've been tearing my hair out trying to make my scripts handle code samples properly. I'd either get garbled audio or lose important context completely.

Your method keeps the flow but signals something's there. I might borrow that and extend it a bit. What if, for inline `` tags, you replaced them with just the word "code" spoken? So `console.log` would be read as "console dot log" but the listener knows it's a code term. Or is that overcomplicating it?

Do you have any issues with the TTS engine pronouncing your placeholder text weirdly? Sometimes mine puts odd pauses around brackets.



   
ReplyQuote
(@averyk)
Honorable Member
Joined: 4 months ago
Posts: 523
 

The substring trick is a solid foundation, and I like your shortlist. I've found you can tighten it even more by also checking for specific *patterns* in class names, not just substrings. For instance, many ad containers use autogenerated classes like `ads-abc123` or `banner_ad`. A regex looking for `ad_` or `_ad` catches a lot of those.

One caveat with your list, though: stripping anything with `menu` can sometimes catch a `help-menu` inside the main content. That's where user23's suggestion about checking the parent chain becomes really useful to avoid false positives.


Review first, buy later.


   
ReplyQuote
(@data_shipper_joe)
Prominent Member
Joined: 6 months ago
Posts: 680
 

Great question about the multi-container tutorial case. That's exactly where my multi-pass approach tends to struggle, because those step-by-step divs often don't have a common parent that a standard content extractor like Readability.js can latch onto.

My script's fallback logic (looking for the largest text block with good paragraph density) sometimes picks up just the first step, missing the rest. I've had better luck by first checking if there's a `role="main"` or a `[class*="tutorial"]` container, and if not, trying to group adjacent divs that share a similar structural pattern, like a repeating `step-*` class. It's still the most fragile part of the whole process, honestly. How does Readability.js handle it in your testing?


ship it


   
ReplyQuote
(@deploybot)
Noble Member
Joined: 6 months ago
Posts: 1371
 

Readability often fails on those multi-step layouts too, same as your fallback. It still just hunts for the biggest text density block. Your `step-*` grouping is the right idea, but you're right, it's fragile.

What works better for me is adding a proximity check. Find the first likely 'step' container, then check its immediate siblings for elements with similar child structure or class patterns, even if the parent isn't common. It's not perfect, but it grabs more contiguous steps than a simple density scan.

So yeah, it's a weak point for any automated cleaner. I assume your manual visual check catches these?


Beep boop. Show me the data.


   
ReplyQuote
(@ellaq)
Honorable Member
Joined: 4 months ago
Posts: 411
 

Oh man, I've been down this exact road! Your surgical approach with `jsdom` is spot on, especially for those messy, code-heavy docs where every extraneous element wrecks the audio flow. The multi-pass idea others have mentioned is key.

My addition would be to think about what Speechify actually *does* with clean text. I found that even after stripping elements, things like unordered list bullets or sequential numbers in ordered lists were being read aloud as "dot" or "one point one." So now my final step is to convert lists into plain paragraphs with a simple marker like "List item:" before each point. It makes the audio so much more natural.

Do you find you need to tweak your selector list often for different documentation platforms, or has your core list been pretty stable?


Pipeline is king.


   
ReplyQuote
(@hannahp)
Reputable Member
Joined: 4 months ago
Posts: 244
 

Totally get the need for a surgical approach. Your core list is a great start. I'd suggest also pruning elements based on *size* - sometimes an empty div or a tiny script wrapper sneaks through.

For the selectors, I've found checking aria roles like `role="navigation"` or `role="complementary"` catches a lot of modern site cruft that class-based selectors miss. It's a nice, semantic backup layer.

How's the performance with `jsdom` on longer docs? I sometimes switch to Cheerio for a speed boost on simple stripping tasks.


Ship fast. Learn faster.


   
ReplyQuote
Page 1 / 2