We looked at over 1400 responses to the 2016 OSMI Mental Health in Tech survey, and this is what we found.
Open Sourcing Mental Illness (OSMI) is a non-profit corporation dedicated to raising awareness, educating, and providing resources to support mental wellness in the tech and open source communities. Every year, OSMI conducts their Mental Health in Tech survey, aiming to measure attitudes towards mental health in the tech workplace and examine the frequency of mental health disorders among tech workers.
We chose to analyze the 2016 OSMI Mental Health in Tech survey (full dataset available on the OSMI website and Kaggle, as the survey questions and number of respondents changed between years, and the 2016 survey had the most respondents overall (1,467 participants).
Our goal was to answer one question: how comfortable are tech workers talking about mental illness in the workplace?
OSMI logo copyright © 2015-2020 Open Sourcing Mental Illness Ltd, and is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
First, we limited the responses to those which indicated they weren’t self-employed, since we were interested in learning about mental illness in the context of workplaces where employees had a direct supervisor.
Then, for open-ended response fields we either consolidated (in the case of gender, which we separated into male, female, and gender minority groups) or omitted (in the case of questions asking the respondent to describe their reasoning or experience).
Finally, we separated multi-response categories, such as those picking the respondent’s type of work from a drop-down list, into binary components, where each possible response became either a “yes” or a “no.”
We chose the question “Would you feel comfortable discussing a mental health disorder with your direct supervisor(s)?” as our response variable, as we determined that this question most closely resembles the question that we are attempting to answer.
Summary of transformed data. Mapping of survey questions to column headers can be found on repository.
We chose to use a logistic model to predict whether a person definitely felt comfortable talking about a mental health disorder with their direct supervisors, answering “Yes” on the survey, or possibly did not feel comfortable doing so, answering “Maybe” or “No” on the survey.
AIC model during first step
AIC model after last step
The first model we built was fit with the entire 34-variable dataset. While it passed the model utility test, we noticed that many variables had extremely large p-values, and thus did not possess much predictive power. Because of this, we then ran the model through an AIC optimization function on R, weeding out the least helpful predictors one at a time until the model with the lowest AIC was the current model. This removed 23 weak variables from the model, for a total of 11 variables in the smaller model.
This new model passed the model utility test. We then performed a nested models test on this model, comparing it to the full model, and found evidence that we could safely remove the weak predictors and use our smaller model for prediction. Finally, to confirm that the model was quality, we ran the Hosmer-Lemeshow goodness of fit test on it and found that it passed, showing that the model adequately fit.
Now that we had an efficient model that had utility and fit well, we then wanted to test its predictive power. To do this, we first performed an 80/20 split, dividing the dataset into training and testing datasets. We then fit the model using the training data, and then used it to predict whether the tech workers in the dataset were likely to feel comfortable talking about a mental health order with their direct supervisors or not.
We found that this model predicted correctly on the test data with an accuracy of 83.9%. This is significantly larger than a random prediction of 50%, showing that by using this model, one can reliably predict whether a worker feels comfortable talking about a mental health order with their direct supervisors or not. In turn, this indicates that the inferences that we can draw from our model have a higher chance of being relevant.
These models and analysis are only valuable insofar as they inform actionable steps towards meaningful workplace reform. Here, those steps are informed by significant and insignificant trends working together to paint a complete picture. First, we see that company structure (size, location, etc.) is not to blame for any significant correlations with mental health. Companies cannot credibly make sweeping statements like “problems are inevitable for all businesses of this size” or “every corporation around here deals with things like this” to justify non-responses to important issues. Any tech company of any makeup can create a healthy working environment, given they put substantial effort into effective reform.
What does effective reform look like? To start, we can consider what doesn’t work. Powerpoint presentations, posters, formal presentations and other token acts of mental health awareness were found to be some of the least significant factors in improving the comfort of employees in discussing mental health. Calling workers into a board room on an arbitrary Tuesday, making them listen to an hour long talk from a HR rep, and sending everyone on their way was never going to solve anybody’s problems. Regardless of intentions, these shallow responses likely frame companies as doing the bare minimum to appear responsible, only caring about employees’ wellbeing as little as they must to avoid widespread backlash. For an employee to genuinely feel comfortable openly discussing mental health illness, they must feel recognized and cared for as a human being.
To adopt a healthy mindset of concern and accomodation, companies need to show employees that it is safe for them to speak up and that their concerns will be met with a meaningful response. To aid with the first point, companies can create a space of trust between coworkers. Feeling comfortable discussing mental health issues with coworkers was one of the most significant determinants of whether an employee would feel comfortable discussing with their supervisors. To promote these spaces of trust, companies could implement small group breakouts that give employees the chance to get to know and build relationships with each other. If an employee can rely on at least one other coworker for support and backup confronting any problems they face, they will be much more likely to reach out for help.
Additionally, companies need to have robust accommodations for employees that do reach out with problems. Companies that respond to mental health issues as seriously as physical health issues make employees much more likely to be comfortable in discussion of their problems. Conversely, employees are much less comfortable when they know or fear that they won’t be given a leave of absence to deal with their mental health illness. Basically, an employee will only reach out to discuss if they think the company will take them seriously, recognize the true severity of their problem, and offer them the correspondingly appropriate time and space to seek help, recover, and adapt to the challenges of their illness. If these conditions are not met, an employee is more likely to think they would be ignored or even threatened if they tried to bring up their condition.
There is no silver bullet for improving mental health in tech companies. Building trust among employees and inspiring confidence in serious mental health accommodations takes time. This is all the more reason why slogans and promotional videos will never substitute attitude changes and mindset shifts. Each policy change is only a single step in the long process of centering company values towards the respect of human dignity. Inevitably, a commitment to this process will require significant sacrifices and reevaluations for many companies. Despite these concerns, we believe that the data clearly shows the effectiveness of correctly implemented human and trust oriented reform in creating a healthy workplace. We sincerely hope then, that all companies realize how the benefits of such reform far exceed the costs.
 About OSMI: https://osmihelp.org/about/about-osmi
 Mental Health in Tech Survey 2019: https://osmihelp.org/blog/2019-mental-health-in-tech-survey-now-open
All code and data presented here is available on this project's GitHub repository.