The Web Scrape component allows you to pull visual content from a webpage and automatically insert it into your emails as a static image, and keep the content within the template fresh and relevant.

• The system scrapes a chosen webpage every hour.
• Each scrape generates a new static image, which is updated in the email at the time of open.
• If the webpage is removed, the most recently scraped image will still display to ensure continuity.

Key Features

• Hourly Updates: Ensures your email content stays current.
• Static Image Output: Consistent display across all ESPs and devices.
• Custom Timestamp: Optionally display when the scrape was last run.
• Built-in Fallback Behavior: If the page is removed or there are problems accessing the content, the last cached scrape is displayed.

Common Use Cases

• Hero Banner Sync: Using the web scrape component, you can scrape the hero banner on your site and include it within your email campaigns, creating a seamless cross-channel experience.
• Leader Board in email: If you have a leaderboard of teams or stats on your site, using the Web Scrape component, you can keep customers up to date on the latest standings by grabbing this content directly from your site.

How It Works

1. A section of a webpage is selected as the scrape source.
2. DynamicMail captures that section and converts it into a static image.
3. Every hour, the image is refreshed with the latest scraped content.
4. This image is embedded into your email via a generated embed code.

Setup Guide

To configure the Web Scrape component:

1. Create the component
   1. Navigate to “Create Component” and select “Web Scrape”.
   2. Enter a unique Component Name so it's easy to recognize later (e.g. “Hero Banner Scrape”).
2. Configure the settings of your Web Scrape 

   

   1. URL to Web Scrape
      • This is the URL that the component will access to scrape content from.
      • Ensure that the full URL is entered including HTTP/HTTPS or required query parameters.
   2. Select Web Scrape Area
      • Clicking the “Select Web Scrape Area” button launches a modal that gives you control over the location and size of the area that the component will scrape.

      

      • You can either drag and manually resize the red box over the area that you want to scrape or enter the exact height, widget and offset coordinates that you want to scape.
      • When you are satisfied with the scape area, click the “save” button to close the modal.
      • Notes on selecting your scrape area:
        • Clicking “Cancel” or outside the modal will close the modal without saving your changes.
        • Everything within the bounds of the outer red box will be included in the scape.
        • Setting up this web scrape area is defined based on coordinates, meaning that if the structure of your content changes it's possible that the content you want to scrape can move out of the scraped area.
   3. Timestamp
      • You can optionally decide if you would like to include a timestamp within the scraped content.
      • Checking this feature will open settings for:
        • Timestamp position and padding
        • Border thickness, font, color, and size
        • Time zone
   4. Click-through URL (optional)
      • Here you can include a CTA link with your scraped content.
      • When a URL is included the component will track both open and click events.
   5. Responsiveness Type
      • Fluid: Stretches to fit screen width.
      • Fixed: Maintains the same width across devices.
   6. Set image to display block
      • In some email clients or templates a small gap can appear below the component and the next element in the email template. Ticking this feature adds a small bit of CSS to fix this gap.
   7. Component Alignment
      • Select if you want the component to align left, right or center.
   8. Alternate Text (optional)
      • If you would like to include “Alt Text” on the component you can enter the text that should be included.
   9. Tracking Parameters (optional)
      • Optionally include dynamic tracking parameters on your click through URL by entering them here. See this article for more details.
   10. Advanced Settings
       • Delay
         • By default the web scrape loads the page and scrape the content immediately. This setting allows you to set a specific number of seconds the scrape should wait before it scrapes the content. This can be helpful if the site loads slowly, contains heavy content or has lazy loading.
         • This delay is only for the scraper to get the content it does not impact the experience of the user viewing the content in email.
       • Dismiss pop-up modal
         • Many sites include cookie pop-ups that block the content, if the cookie pop-up is able to be dismissed by clicking outside the pop-up. By ticking this box before the scraper scrapes the content it will click in the upper left corner of the page to try and close the pop-up.
         • Our recommendation is to use the Web Scrape on pages that do not include these cookie pop-ups or overlays.
       • Viewport Width and Height
         • By default the web scrape loads the web page in a full size desktop browser view port and takes a screenshot from that content. If you want to extract content as it appears on a smaller screen, you can use these fields to control the screen size that the web scrape is using to access the Web Scrape URL.
3. Click Update Preview to see a live preview
   • Once you have set up your component settings, click the Update Preview button to render an updated preview of your web scrape component to see what your web scrape will look like.
4. Click Generate Code to create an embed code
   • If you are using the OptiMail content block, click the “insert in template” to place the component into your template.
   • If you are using an other ESP, you can copy the provided HTML code and paste it within your email template.

Recommendations & Limitations

• The component captures a selected area of a webpage and converts it into a single, static image. Because the final output is a static image, the tool cannot capture interactive or moving elements like videos, GIFs, or carousels within that area.
• The scrape area is based on set coordinates. If the webpage layout changes, the scrape area will not adjust automatically.
• Avoid scraping content that relies on cookies, pop-ups, site interaction, or custom JavaScript.
• Scraping third-party sites with such features may cause issues.
• The web scrape component cannot access content that requires a sign in.
• The web scrape component is limited to 1200px by 1000px.

Compatibility

• This component is fully supported; no fallback required.

Reliability & Error Handling

• If the target page is removed or the scrape fails to load, the last successfully scraped image displays.
• If there are any changes to the layout of the content on the page, this can affect the render of the web scrape. This may be seen as content being cut off.

Security & Compliance

• The web scrape component is intended for scraping your own content.
• The component is not designed to access password protected or regionally protected content.
• Our component uses US-based servers by default. If your content is geographically restricted, your IT team may need to whitelist our static IP addresses to allow access.
  • US IP Addresses: 35.199.60.237, 35.221.56.26, 35.221.33.157, 34.86.85.20, 35.221.48.107
  • EU IP Addresses: 35.242.199.57, 35.198.188.17, 34.107.29.24, 35.246.154.200, 34.107.86.58

Frequently Asked Questions (FAQ)

Custom Development Options

Some scenarios require DynamicMail team involvement and may come at an additional cost:

Dynamic Links

By default, the click-through URL is fixed. We can help create scrapes where the URL updates automatically to align with the changing scraped content (e.g., a link to a new article).

Carousel Display

We can help scrape multiple pieces of content and embed them into an interactive carousel, including dynamic URLs for each item.

Adapted Web Scrape

This allows for modifying the appearance of scraped content. For example, scraping a block of text but changing its color or excluding an icon in the final email image.