Home › Forums › NodeXL Data Importers: Issues with Twitter, Facebook, YouTube and other network importers › Problem getting data? Importer issues
Tagged: Access, Data, Importer, Social Media, Social Network
- This topic has 35 replies, 3 voices, and was last updated 2 years, 5 months ago by
Marc Smith.
- AuthorPosts
- December 4, 2017 at 5:21 pm #7571
Marc Smith
KeymasterHave an issue getting the data you need? Share the details here!
December 12, 2017 at 9:19 am #7586Rajat_Sharma
ParticipantYes. i need to import facebook friend network data. Please suggest how it can be done.
Thanks
Rajat SharmaJanuary 24, 2018 at 7:34 am #7712mustafayavas
ParticipantHi,
I am using “Twitter Users Network Importer” and I am trying to generate the network of “follows” relations of a set of usernames. I am curious what exactly the option “Limit friends and followers to: X” serves to, when I choose “Basic network plus friends”.– Is it the maximum number of followers that the app samples from the followers of each specified users? (e.g. Say I am checking @XYZ who has 5M followers, and if I limit to 5000, then it will return a random sample of 5000 followers of @XYZ. OR is it going to look at a random sample of 5000 followers and followings of @XYZ?)
– Is there an upper limit to this?
– To put the question more generally, when I choose “Basic network plus friends”, how does Node XL generate “follows” relations? Does it send of query to the list of followers of each users or to the list of following of each user, or both?
– When I choose “Import only the Twitter users I am interested in”, does Node XL limits to its queries only to the list of usernames I provide? In other words, when it is chose, does Node XL query only whether any two specified users follow each other or not, OR does it query the whole list of followers node 1, and checks whether node 2 is in the list or not?
I have searched all over but could not find a satisfying/precise answer to my questions.
Thanks,
mustafaJanuary 29, 2018 at 10:00 am #7716Marc Smith
KeymasterNodeXL Pro can import from Twitter in two ways. The search importer collects tweets containing a particular term. The Users import collects tweets from specified users.
You may apply multiple queries into the same workbook by changing the settings in the NodeXL Pro> Import> Import Options dialog:
Uncheck the option that reads:”Clear the NodeXL workbook before the data is imported.”
You may then import multiple data sets into the same workbook.
Regards,
MarcJanuary 29, 2018 at 10:02 am #7717Marc Smith
KeymasterHello!
Facebook’s API no longer enables access to the personal friends list.
Facebook Pages and Groups are accessible.
You may import network data from Facebook using the:
NodeXL Pro> Data> Import>Improt from Facebook Page
NodeXL Pro> Data> Import>Improt from Facebook Group
Importers.Regards,
MarcFebruary 5, 2018 at 7:30 am #7740Jim43215
ParticipantRecently, I have encountered an issue using the Twitter Users Network import.
When an import is executed, the following error message appears “Either there are no such users, or they have all protected their Twitter accounts”.
This import was run ~3 months ago, using the same usernames. e.g. @cbsnews
The error occurs regardless of searching basic network only or basic plus.
Note: This error does not occur when importing via the Twitter Search Network.
Any suggestions would be greatly appreciated!
February 5, 2018 at 8:17 am #7741Marc Smith
KeymasterHello!
Thanks for the interest in NodeXL!
Twitter’s API data volumes can vary based on factors internal to Twitter.
NodeXL will execute the same calls every time, but Twitter’s results may vary.
We do find that in some cases a delay of 24 hours will return the volumes delivered by Twitter to higher volumes.
Only a commercial feed from Twitter can guarantee a sustained rate of data. These services are currently very expensive.
If you would like to share the query you are running, I can try to replicate it here – perhaps I will get more data?
Regards,
Marc
February 6, 2018 at 9:25 am #7742OscarThynne
ParticipantIs it possible to find out all of someone’s followers and then the interactions between the followers and individual of interest?
February 6, 2018 at 9:55 am #7743Marc Smith
KeymasterIt is possible but it is also likely to be slow.
What are the limits of the Twitter data importers?
The use of NodeXL Pro Twitter data importers requires a twitter account. Before your first data import you need to authorize NodeXL by entering a token number which is automatically sent to you during the authorization process.
Moreover Twitter’s public free API has many limits. NodeXL Basic and Pro are both effected by these limits. Data is available only for 8-9 days. Queries cannot return more than 18,000 tweets.The follower network is further restricted: the rate at which queries about who follows who can be asked is low.
NodeXL Pro does not enable the collection of data beyond these limits. That said, NodeXL can process data from commercial data providers (like Crimson Hexagon or Radian6). While these are expensive options, they may be the only way to get historical data from Twitter.
Commercial services like Radian6 and Crimson Hexagon provide archival data – but not cheaply!
You may be able to get a little bit more data from the public Twitter API by using the SINCE: and UNTIL: operators – example:QUERYTERM since:2016-01-21 until:2016-01-27. Since: and Until: operators scope the time frame of the query.
Twitter controls its API and throttles it based on unknowable parameters. We notice that the more the volume of tweets == less tweets delivered. One alternative is to do day long slices and append them in order to maximize the data available from Twitter.
You may also be interested in th Connected Action Graph Server Importer which enables NodeXL to connect to the “STREAM” API from Twitter (which sometimes delivers larger volumes of data).
Twitter applies additional limits to the Follows and Followers data query — it is “throttled” to a slower rate of data delivery than the other “Tweet” based queries.
If you are patient it may be possible to do this collection. That said, it is often the case that one or two users in a list of followers are celebrities who have very large numbers of followers. It is not practical to collect many tens of thousands of followers in a desktop application. Millions are not viable at all. NodeXL Pro offers the ability to set an upper threshold (say 5000) on the number of followers that will be collected. Note that the typical Twitter user has about 200 followers (many of which are bots or spam accounts).
In practice, the Follow network may not be as userful as the “SEARCH” network based on an active “Mention” or “Reply” from one user to another. A “Follow” may not indicate any awareness or exposure to the other user’s content. In contrast, Reply and Mention are stronger indicators of content exposure and engagement.
February 7, 2018 at 4:23 pm #7744Sandro
ParticipantHi Marc, today I’m having trouble downloading Facebook Fan Page Data with NodeXL Pro. This is a problem that I didn’t have yesterday.
Do you know what’s going on? Thanks!February 7, 2018 at 5:17 pm #7745Marc Smith
KeymasterThe recent changes to the Facebook API have broken some of the features of the NodeXL Facebook importers
https://developers.facebook.com/docs/graph-api/changelog
We are investigating and will release an updated version of the importers soon. It is not yet clear how much of previous functionality will be retained depending on what remains of the key API calls.
Regards,
Marc
February 8, 2018 at 2:17 am #7746ayke
ParticipantHi Marc,
I keep getting this error message when downloading Facebook Fan Page Data using NodeXL Pro:
The network couldn’t be obtained. Details:
[KeyNotFoundException]: The given key was not present in the dictionary.
The last time I download fanpage network data was at February 2nd 2018. Does this happened because the recent changes of Facebook API like you mentioned above? If yes, then could I still get facebook fanpage network data in the future? When will the new NodeXL Pro updated version be released? I also would like to know why I never get user gender information when importing facebook fanpage network.
Thank you
February 12, 2018 at 1:06 pm #7752Marc Smith
KeymasterThank you for the interest in NodeXL.
Facebook’s API no longer enables access to the personal friends list.
Facebook Pages and Groups are accessible to a limited extent.
You may import network data from Facebook using the:
NodeXL Pro> Data> Import>Import from Facebook Page
NodeXL Pro> Data> Import>Import from Facebook Group
Importers
The recent changes to the Facebook API have broken some of the features of the NodeXL Facebook importers
https://developers.facebook.com/docs/graph-api/changelog
We are investigating and will release an updated version of the importers soon. It is not yet clear how much of previous functionality will be retained depending on what remains of the key API calls.
It looks like the Fan Pages are more affected than the Groups importer.
The Fan Page importer looks like it no longer providers userIDs or reactions.
Many of the “edges” NodeXL extracted from Facebook were based on these data elements. These edges will no longer be available due to the Facebook changes in the API.
We are exploring ways of expanding on the existing Facebook Fan Page to Fan Page network collector – which still looks like it is available.
Facebook is a commercial property and is not obliged to provide *an* data if it does not want to.
That said, these recent changes to reduce data access reduce the credibility and desirability of Facebook as a platform for many kinds of users.
I predict that a Premium Facebook “product” will soon emerge to sell back the data that we all contributed to Facebook for free.
February 25, 2018 at 12:31 am #7791angel
ParticipantHello, today, i’m have a problem in the facebook fan page import functionality. The error message is [KeyNotFoundException]: The given key was not present in the dictionary.
There as any solution?
Thanks
Angel
March 5, 2018 at 3:27 am #7792Durman_Ludovit
ParticipantHi,
I have trouble with Facebook social media importer, i can´t get data about likes and shares from fan page network, only comment data are available. See here: https://imgur.com/4AdDBss Are this data lost forever?
Thanks
L.
- AuthorPosts
- You must be logged in to reply to this topic.