Community support plays a crucial role in the effectiveness of prosecutorial justice sector and law enforcement. However, measuring public sentiments towards relevant policy implementations is difficult. Conventional methods are labor-intensive, time-lagged, and not scalable. Alternatively, we investigate the viability and strategies of leveraging social network services to assess public sentiments in a massive, timely, and effective manner.

Focusing on Twitter, we first evaluate strategies to harvest relevant tweets and mitigate contamination. This includes keywords selection and streaming pipeline modification. Then, using the collected tweets regarding prosecutorial justice departments at both local and national levels, we assess whether a sentiment baseline exists based on the positive, neutral, or negative sentiments perceived by existing Natural Language Processing models. Finally, we investigate geographical and longitudinal variations in sentiments.

The results show that measuring such sentiments is challenging.

• First, Twitter’s free Streaming API is not ideal qualitatively nor quantitatively for collecting the desired data.
• Second, the data is highly contaminated due to the inevitably ambiguous keywords.
• Finally, most tweets appear either neutral or slightly negative in the two models respectively.

However, it is possible to observe and compare longitudinal trends and changes instead of absolute values. Recommendations for similar research pipeline are then concluded.

Based on our results, concrete baselines for public sentiments towards prosecutorial justice sector are currently unavailable through tweets. However, changes in public sentiment can be observed. This may serve the needs of assessing community responses to events and policies. While our results showed most changes were within the margin of error, this does not rule out the possibility that statistically significant changes in sentiment could be detected.

One limitation we faced was that we could not acquire a search API from Twitter, limiting use in three ways. One, we were limited in the number of tweets to 1 per thousand. Two, we could not set restrictions to tweets containing a combination of search terms. Three, we could not search historical tweets, limiting our sample to an 8-week period. With a search API or firehose, it would be possible to get a larger and better quality sample that would reduce the error range of the sample and having a longer time period of data would increase the probability of some change in sentiment occuring during that time.

We suggest following recommendations for conducting a sentiment analysis in similar topics. First, the strategy can vary by the type of Twitter API available. However, unless the API is unlimited, one may optimize the query using a Streaming API before consuming the quota. The keywords associated with this topic are inevitably general. One needs to check the retrieved tweets and update the keywords addressing major contamination. If operating with Streaming API, one may use more specific keywords (e.g., names of the program or event) to mitigate excessive irrelevant tweets. With a purchased Search API, one has the privilege to use the intersection operator and geotag information for more precise queries.

This is a collaborative project between CUSP and the sponsor BetaGov. BetaGov has been working with prosecutorial justice departments across the United States to implement innovative solutions. One example is the police-to-peace initiative that includes ”peace officer” on patrol cars, expecting to improve community sentiments towards law enforcement units. The arising question is to know if these implementations are taking effect as expected.