The Border Observer - Tedious, Mundane, and Boring Details

Picture grilled vegetable and chicken brochettes, pxhere.com/CC0 I'll cover this more later in this blog, but in short — Twitter, back in June, forced all developers to reset their access tokens. The underlying purpose was to limit developers to about 1500 tweets per month, and if you went over the limit - your access was "cutoff" until the end of the cycle (in our case, the remainder of the month) _or_ you could pay Twitter money to tweet more. The good news is that "cutoff" ends in a day, namely June August 6 at midnight. After that we will adhere to the 1500 tweet per month limit. More good news after this next section. The remainder of this blog is for the concepts to the next blog. In the next blog I will discuss the moving target of "journalism" and inner workings of the Search Engine and it's Results Page . However, the next few blogs are to make the point that new tools are sorely needed. In addition, the current practices and meth

  Picture Screenshot of Morelos Dam gate in Yuma Arizona, Newsnation/Ali Bradley On 24 Feb 2021 and 23 Apr 2023 , two blogs addressed the direction of #BorderObserver. To recap, our goal has always been to build robots - for the southern border and other domestic tasks. At the time we noted that building robots would be put off for two (2) years. In the meantime, we've refocused several times, but always kept something going on twitter with @BorderObserver The New Focus  A year ago we realized that we made a product mistake with a database for press releases. The less than luke-warm welcome informed us that we missed the mark. The new focus has been a SEARCH ENGINE. The Search Engine  A common mistake by consumers is that Google or Bing are search engines. That is not the case for either. What both have done is collect common-internet connections and found the associated words that go with each connection. NEXT, and this is their key, both customize the results you see based on w

 Every day three (3) tasks require overnight attention Update the Search Engine results (DAILY: DL 15m/Process 1+hr) Scrape 70+ CBP Twitter accounts for text, pictures and videos (DAILY: DL 1+hr/Process 2+hr) Download CBP Weekday Press releases (WEEKDAYS: DL 15m/Process 1-3hr) This does not include maintenance of in-house software used, random tweets added, and 8-12 hours of videos and podcast playing in the background. Four (4) NEW THINGS Soon we will restart processing USAO (United States Attorneys' Offices) press releases. Details as the process restarts. Soon our search engine website will have a "dedicated" webpage for links to CBP press releases. Details as the process rolls into place. In a few days, we will describe - in detail - how our search engine works and how you can help. On the near horizon is access to CBP Tweeted Images. Details below TL;DR (Too Long; Didn't Read) Daily, #BorderObserver will scrap 70+ CBP twitter accounts for the text, images and vid

 If you have not read the original announcement for the SE (Search Engine), here is the link . The title is Our First Blog Post and Our NEW "News Search" Engine. In case you missed it, the purpose of SE is that the READER determines the order of sources, categories, and search words. IF you are looking for the test page of the SE click here or the screenshot below The SE is now being updated daily. More below the image. To be clear, THIS IS NOT THE BETA. This is a working visual prototype. When we go into BETA, the page will be updated daily for a *few* days. Then it will update several times a day ― as news gets updated. The previous day's news will be at the bottom of the page. Below is a screenshot of the BETA in development. More below the image. Within a month or so, the SE will have a dedicate website with many more options. Right now, the SE only does the most basic tasks — namely, display the pre-sorted headlines and pops open a webpage to the link. That'

Today, we are announcing our new "Border News" Search Engine (SE). It is not your typical search engine. In a few days, the first draft version will be available publicly. The SE is presearched , preindexed and reviewed by professional working in the area. Sources are determined by the reviewers. Currently, there are 25 sources, but this will quickly grow. If you are a reader, here is how it will work: You do not need to sign up or give up any personal information. "Border News" SE is updated daily. The news headlines are organized by categories or topics. Sources are organized by the reviewer. Reviewers get a highlighted spot on the list for the items they wrote or created. As the reader, your preferred sources , categories or topics with your keyword searches are placed before any of these. You determine the order of sources, categories, and keywords. After your preferred lists, then the general results of search engines (the main page)  are announce