Welcome! This forum has a treasure trove of great info – Scouters helping Scouters! Just a heads up, though - all content, information, and opinions shared on this forum are those of the author, not the BSA.
BOY SCOUTS OF AMERICA

Scouting Forums

Python Requests is blocked from accessing Scoutbook

I am creating a website for my troop, and I would like to embed the Scoutbook calendar in the site. Since iCal links are provided, I found this utility to display the upcoming events. However, it uses the Python requests module to get the calendar information. After it gave a few errors that I then investigated, I have concluded that Scoutbook is blocking HTTPS requests that have a user agent of python-requests. I understand that this may be to block crawlers or denial-of-service attacks, but the module allows the User-Agent header to be changed, making for an ineffective filter. My request is for this filter to be removed since it can easily be circumvented and there are plenty of use cases where a little automation can make for a more seamless user experience. Alternatively, it could just be removed for requests to the ics/ directory so that more calendar integrations are possible.

Below is a sample of code that doesn’t work due to the filter. (The requests module first needs to be installed with pip.

import requests

requests.get("https://www.scoutbook.com/") # Returns a 404 response

@AlanEverett - the calendars within scoutbook are natively behind the login. Scoutbook.com is just a landing page. The ics is unit linked so I am not certain that you would be able to gain much from this ics:

https://www.scoutbook.com/ics/206734.7370F.ics

So I am not really sure this is a bug, or something not possible due to the structure of scoutbook itself.

I dunno, @Stephen_Hornak. I can see your crew’s calendar by importing the ICS file you linked to.

There is nothing specific to the ics files that are being blocked, just that Scoutbook doesn’t allow the python-requests user-agent. Getting a calendar set up is just my end goal. If I use a different module to make the HTTPS request or configure requests with a different user-agent string, I get the response I need. However, I don’t have control over the code in the calendar utility that I found.

Thanks Charlie - the ics makes it public then. My mistake…a bit tired today I guess.

I again would not think to classify this as a bug. We use a Google calendar insert into our web sites which are all Google based.

Is the web site going to be maintained by the scouts via the webmaster position?

I was debating between the Bugs and Feature Requests categories since it seemed to be somewhere in-between. I ended up choosing Bugs.

I tried using Google as an intermediate, but it wouldn’t work due to the iCal being configured as a private link. (Technical distinction; I’m not really sure why it matters.) If you have any suggestions for getting it successfully embedded, that might work too.

I would assume so. I’m my troop’s first webmaster, so I’m kicking off the site.

@AlanEverett - are you a scout or adult in the unit ?

I would highly doubt that you will get scoutbook code modified to satisfy a python call.

Now as far as using g-cal, we added the ics to unit Google calendar then inserted that into the site. If you go to scouting33.org the links are there for all three units. The calendar was then made public.

On my first question is based on the legacy this would leave behind. Something this complicated would not easily get put to use by youth or many adults.

I’m a scout, although probably more technically proficient than most.

I can’t claim to know how the internal workings of Scoutbook work, but I assume that this is probably something in how the server was configured. I suspect that there is a list of blacklisted user-agents, so I am only asking for python-requests to be removed from said list. If I had a server that I could host a utility on, I would just adjust the code to use a different HTTP client, but I don’t.

As I said, I tried using Google as an adapter with the following code, but it just gives the error in the next snippet in the console:

<!--Ignore this comment, it just makes the formatter work-->
<iframe src="https://calendar.google.com/calendar/embed?height=600&amp;wkst=1&amp;bgcolor=%23ffffff&amp;ctz=America%2FNew_York&amp;src=c251MHBkbWs2YmhyOXNpZmhscmJzN2pyOTA2YXU5dXNAaW1wb3J0LmNhbGVuZGFyLmdvb2dsZS5jb20&amp;color=%23E67C73&amp;showNav=1" style="border:solid 1px #777" width="800" height="600" frameborder="0" scrolling="no"></iframe>
Refused to display 'https://accounts.google.com/ServiceLogin?service=cl&passive=1209600&osid=1&continue=https://calendar.google.com/calendar/embed?height%3D600%26wkst%3D1%26bgcolor%3D%2523ffffff%26ctz%3DAmerica/New_York%26src%3Dc251MHBkbWs2YmhyOXNpZmhscmJzN2pyOTA2YXU5dXNAaW1wb3J0LmNhbGVuZGFyLmdvb2dsZS5jb20%26color%3D%2523E67C73%26showNav%3D1&followup=https://calendar.google.com/calendar/embed?height%3D600%26wkst%3D1%26bgcolor%3D%2523ffffff%26ctz%3DAmerica/New_York%26src%3Dc251MHBkbWs2YmhyOXNpZmhscmJzN2pyOTA2YXU5dXNAaW1wb3J0LmNhbGVuZGFyLmdvb2dsZS5jb20%26color%3D%2523E67C73%26showNav%3D1&scc=1' in a frame because it set 'X-Frame-Options' to 'DENY'.

If it helps, I’m trying to make this site using Wix since it offers more features than Google Sites.

While I see where you are coming from, I am not directly using the python script. I am trying to use the Open Web Calendar utility, which is open-source and coded in python. I just happened to have the technical know-how needed to look into why it wasn’t working, and I came to the conclusion that it is an issue on Scoutbook’s end.

The utility just acts as an intermediary to convert from ICS to HTML so that web browsers are actually capable of viewing the file. It generates an iframe-compatible link from multiple iCal links. In addition, I feel that the calendar produced by the utility actually ends up looking nicer than what Google lets you embed.

@AlanEverett - I applaud your effort and wish you best in building the site. As I noted I am not certain that the BSA will entertain a change in servers or code to allow your code to work. My opinion but I can not see the BSA yielding to that request. Now again I remind you of legacy…a bit down the road when the BSA moves the calendar to a module on the new stack what will happen with your code. Will that 12 or 13 year old that is now webmaster be good to go… I am not sure. I will agree that the new g-sites lack features, but in the end a scout can manage it and it serves its purpose.

Well, I was successful in getting a Google Calendar to cooperate since I missed the “Make the calendar publicly accessible” checkbox when adding the iCal to my account. However, I still believe it would be good to allow the calendars to be accessed programmatically, possibly even making them a cross-domain resource.

Bump. Still hopeful that this will be considered. At the very least, I would like to know an official reason why it won’t.

I used a workaround that gets what I need, I created an account on a site teamup.com and import in the .ics files and then embed that into my google sites page. it is not quite live but pretty close,