Polinode

Social Data

The Social Data module allows you to build networks from public social media activity on X / Twitter, LinkedIn and Instagram. You configure a Data Pull describing the profiles to start from and the kinds of interaction to collect, run it, and Polinode purchases the data from third-party data providers and produces a network for you, ready to explore like any other network in Polinode.

Please note that the Social Data module needs to be enabled for your organization before you will be able to use it — it is not available on individual accounts. If you would like it enabled please contact your Account Manager or support@polinode.com. Once enabled, a Social Data item will appear in the left-hand side menu. Collecting social data consumes Data Credits — see Data Credits and Costs below.

The Social Data Screen

Clicking on Social Data in the left-hand side menu shows a list of your saved Data Pulls. Towards the top of the screen you will see your current Data Credit balance along with an Add Data Credits button, a "Search data pulls" input and a New Data Pull button. The list has the following columns:

  1. Name: The name of the Data Pull. Hovering over the name shows a summary of its seeds and hops.
  2. Platform: The social network the pull collects from.
  3. Interaction types: The kinds of interaction the pull collects.
  4. Last run: The status of the most recent run — queued, running, succeeded, succeeded (capped) or failed — or "Never run".
  5. Modified: The date the Data Pull was last modified.

Each row also has action buttons to Run the pull, view its Run history, Edit it and Delete it. While a run is in progress the Run, Edit and Delete buttons are disabled until it finishes. Deleting a Data Pull removes it and its run history but keeps any networks it has already produced.

Creating a Data Pull

Click on the New Data Pull button and a drawer will open on the right-hand side of the screen. Give the Data Pull a name and select the Social network to collect from — each platform offers a different set of interaction types. Then work through the following sections.

Seed Profiles

The seeds are the profiles the collection starts from. You can supply up to 5,000 of them in one of two ways:

  1. Upload a file: Click Upload Excel to upload a spreadsheet with one profile per row, preferably in a single column headed "Profile URL". A Download template link provides an upload-ready example for the selected platform. After uploading you will see how many valid profiles were found, along with any rows that could not be parsed.
  2. From an existing network: Select one of your existing networks and the attribute column that contains the profile URLs. The column's values are read and validated when you save.

The accepted formats differ by platform: for X you can supply handles (with or without the @) or profile URLs; for Instagram, usernames or profile URLs; for LinkedIn, only personal profile URLs (those containing /in/) are accepted — company and school page URLs are rejected.

Interaction Types

Tick the kinds of interaction to collect. Each becomes a set of edges in the resulting network:

  • X / Twitter: Follows, Mentions, Replies, Retweets and Quotes. (Likes are not available on X.)
  • LinkedIn: Post Comments, Post Reactions, Shared Employer and Shared Educational Institution.
  • Instagram: Follows, Post Comments and Post Likes.

The LinkedIn Shared Employer and Shared Educational Institution types are inferred edges: rather than observing an interaction directly, Polinode compares the employment and education history on each pair of profiles and creates a weighted edge where they overlap. The weighting takes into account how long two people overlapped at the same employer or institution and how recently — see Advanced Settings below. Shared Employer edges also carry a Currently Colleagues attribute (Yes/No) flagging pairs who are presently at the same employer.

Profile Enrichment

Turning on Enrich profiles adds profile attributes to the network's nodes, and you can choose exactly which attributes to include. The available attributes depend on the platform — for example Follower Count, Bio and Location on X; Headline, Current Company and Current Title on LinkedIn; Follower Count, Bio and Business Category on Instagram. Each platform also offers a Profile Image URL attribute which, rather than appearing as a column, sets the profile image as the node's image in the network visualization.

Enrichment on X is free — the profile data is already collected as part of every run. On LinkedIn, enrichment uses the same profile data as the Shared Employer / Shared Educational Institution edge types, so if you have selected one of those types the enrichment attributes come at no extra cost. On Instagram (and on LinkedIn without a shared-history edge type selected) enrichment costs additional Data Credits per profile.

Time Window

Collect interactions from controls how far back the collection goes: only interactions on or after this point are collected and counted. You can choose Any time, one of the rolling options (Last 7, 30, 90, 180 or 365 days — the default is Last 90 days) or a specific start date. Without a window, collection is most-recent-first up to the per-run caps, so busy and quiet profiles can cover very different periods — a window keeps the comparison like-for-like. Interaction types that aren't time-stamped (such as Follows and the LinkedIn shared-history types) ignore the window, and this is noted against those types when you select them.

Expansion

Hops controls how far the collection spreads out from your seeds: 0 collects only edges among your seeds, while each hop expands the set through the profiles discovered in the previous round. When hops is greater than 0 two further settings appear: Min. incoming connections, which requires a discovered profile to be connected to at least this many profiles in the current set before it is added (keeping the expansion focused), and Max profiles per hop, a hard cap on how many new profiles can be admitted at each hop, which also bounds the maximum credits a hop can consume.

Usage Limits

  1. Max Data Credits per run: A hard stop — the run ends once it has used this many Data Credits and the network is built from everything collected to that point (the run is marked "succeeded (capped)"). New Data Pulls default to a limit, and if you clear it a run will reserve your full available balance as its spending limit instead.
  2. Skip follower lists above (X and Instagram, when Follows is selected): Profiles with more than this many followers keep their node and their other edges, but their follower list is not collected. This avoids spending credits on very large follower lists that add little analytical value.
  3. Skip engagers for tweets above (X, when Replies, Retweets or Quotes are selected): Tweets with more than this many total interactions keep their node and other edges, but who engaged with them is not collected — the engagers of viral tweets are mostly passing audience rather than genuine relationships.

Advanced Settings

For LinkedIn pulls that include Shared Employer or Shared Educational Institution edges, an Advanced settings section lets you tune the inferred edge weighting: the Base weight each edge starts from, the Overlap bonus added when two people's time at the employer or institution overlapped, the Overlap bonus half-life (the bonus halves every this many years since the shared period ended, so recent overlaps count for more), and for Shared Employer a Geography factor that discounts (or with a value of 0, removes) edges between people who were never in the same geography. People currently at the same employer are never discounted by geography. The defaults work well for most networks.

Once you are done, click Create (or Save) to save the Data Pull, or Create and Run to save it and start a run immediately.

Running a Data Pull

When you run a Data Pull, a confirmation dialog summarizes what will happen: the run reserves Data Credits up front — the pull's Max Data Credits per run if set, otherwise your full available balance — and refunds whatever it doesn't spend when it finishes. The reservation is therefore an upper bound, not the expected cost. If you don't have enough credits available to cover the reservation the dialog will tell you how many more you need.

The dialog also has a Use cached data where available switch, which is on by default. Where Polinode has recently collected the same data, it is reused at a 50% discount and the savings are reported on the run. Turn the switch off if you want fresh data only, with everything re-collected at the standard price.

Runs happen in the background and can take a while for larger pulls — you will receive an email when your network is ready (or if the run failed, in which case no credits are charged). Each Data Pull can only have one run in progress at a time, and an organization can have at most two Social Data runs in progress at once.

Data Pulls can also be run on a schedule using the Automations module — for example, refreshing a network with the latest data every Monday morning.

Run History

The Run history button opens a timeline of the pull's most recent runs (up to 50). Each run shows its status, when it ran, progress (profile and edge counts, including the current hop while running), the Data Credits spent against the amount reserved, how many fetches were served from the cache and the credits saved by caching, and any error if the run failed. For a successful run you can jump straight to the produced network via its link, or download the run as an Excel workbook using the download button.

The Excel download contains three sheets: Summary (the pull, platform, run timings, whether cached data was used, whether the run was capped, and the credits charged), Nodes and Edges. If the network has been edited since the last download, the workbook is regenerated on the next request.

The Produced Network

Nodes are the collected profiles, labelled by name or handle, with base attributes including Platform, Handle, Profile URL, whether the profile was a Seed and its Hop, plus any enrichment attributes you selected. Where a Profile Image URL was collected it becomes the node's image in the network visualization. Each selected interaction type contributes an edge attribute named after it — for example "Mentions - Total" or "Shared Employer - Total" — along with a grand "Total" attribute summing the interaction counts, which makes it easy to size or filter edges by overall interaction volume.

The usual platform limits of 50,000 nodes and 250,000 edges per network apply.

Data Credits and Costs

Social Data consumes prepaid Data Credits, which are shared with the Research Data module. Credits are priced at $0.01 each. Different operations cost different amounts of credits — for example, collecting a follower list costs far less per record than enriching a LinkedIn profile — and the Max Data Credits per run limit gives you a hard ceiling on what any single run can spend. As a rough guide, $1 (100 credits) collects roughly 16,000 X followers, 3,300 tweets or 100 LinkedIn profiles; the full per-operation rates are in What your Data Credits get you. Credits are reserved when a run starts and unspent credits are refunded when it settles, data served from Polinode's cache is charged at half price, and a failed run is not charged at all.

To add credits, click Add Data Credits on the Social Data screen. Admin users in your organization can have credits added immediately with an invoice to follow, while other members' requests are sent to your Polinode account team. Data Credits never expire. Your balance is shown on the Social Data screen, on your Billing page and on the organization Usage dashboard.

← Research Data →