Fuzzy Set Analysis

Question: In the Raw Data table, no one had a score of 1. The closest I saw was 0.95. Was 0.95 a proxy calibration for 1?

Jodi's Answer: The computer program has a glitch when you put in 1 instead of 0.95. That is the only reason. It does better with 0.95 and 0.05. Lee Green can explain why, I think.

Marcus's Answer: FsQCA [Fuzzy-Set Qualitative Comparative Analysis] was set up to support an approach to calibration (the "calibrate" function) that anchors the fully in and fully out conditions at 0.95 and 0.05, respectively. This may be the reason why this issue arises with this software. The STATA fuzzy module does permit fuzzy membership of 0 and 1.

Question: What single article would you recommend reading to introduce fsQCA to colleagues?

Jodi's Answer: I would recommend the books written by Ragin. There really isn't a good article, but I'm hoping Marcus will write one.

Marcus's Answer: The methods brief we wrote for AHRQ is a good simple introduction to the method. I also recommend Ragin's HSR (Health Services Research) article from about 10 years ago. However, to understand the method well enough to actually use it, I recommend folks read Ragin's most recent book or the just published book by Wagemann and Schneider.

Michael's Answer: I highly recommend the methods brief, which is available at http://pcmh.ahrq.gov/portal/server.pt/community/pcmh__home/1483/Fuzzy-Set_Qualitative.

Question: How should investigators approach writing grants that propose to use QCA, given most reviewers will not be familiar with the method?

Jodi's Answer: I haven't succeeded in that yet so it's hard to answer, but spend time explaining what it is and why it is different. Also, provide examples.

Marcus's Answer: There's still not a lot of evidence here, but doing a good job of explaining what fsQCA is and why it's either better or complementary to more traditional methods for the study at hand. Also referring to the AHRQ methods brief and the growing number of published papers using the method in health services research should help. I would avoid getting into the details of the method and try to keep the focus on why it is a good fit for the specific study.

Michael's Answer: Although it doesn't discuss Fuzzy Set Analysis, the NIH OBSSR [National Institutes of Health Office of Behavioral and Social Sciences Research] Guidance on Mixed Methods Research at http://obssr.od.nih.gov/mixed_methods_research/ may also be helpful.

Question: What are the relative advantages and disadvantages of using this approach vs. a cluster RCT [randomized controlled trial]?

Marcus's Answer: A cluster RCT is, by definition, not good with observational or comparative data. FsQCA is set up to help the investigator make sense of observational case studies.

Question: Can you talk more about your experiences with submitting your QCA papers to journals, particularly HSR and clinical journals?

Marcus's Answer: I have tried to publish two papers using this method, both to HSR. The first was accepted, and the second was rejected because the topic was not felt to be aligned with the current HSR editorial focus. HSR editors encouraged me to submit to another journal. I am now editing the paper for resubmission to another journal. I have also reviewed a paper that was published. I observed from both experiences that reviewers are not familiar with the method and have some discomfort with it because of that; but that does not seem to preclude the paper getting published.

Question: Once we have identified individual necessary and sufficient conditions, are both types of conditions considered during the truth table analysis (TTA)?

Marcus's Answer: One reason to identify necessary conditions is that they don't need to be included in the subsequent truth table analysis, which allows the TTA to focus on other conditions (this is important because it's generally important not to include too many conditions in a TTA). To achieve the best possible coverage, one should try to include all possibly sufficient conditions in the analysis.

Question: How does QCA handle missing data?

Marcus's Answer: –I'm not sure.

Michael's Answer: The answer should be available at http://www.u.arizona.edu/~cragin/fsQCA/software.shtml.

Question: Could you talk a bit more about degrees of freedom, e.g., if you have 10 cases, how many conditions can fsQCA support?

Marcus's Answer: Axel Marx did some work on this with QCA that is probably generalizable to fsQCA. His papers are cited in the AHRQ Methods Brief. In brief, there are some recommended limit ratios for cases/conditions. The more cases you have in your set, the more conditions you can include in your TTA, up to a point.

Question: Do you need to have an in-depth knowledge of the issue because you are biased in the selection process?

Jodi's Answer: You need in-depth knowledge of each condition you are including because you have to calibrate. If you don't know your data well, it will be difficult to do that.

Marcus's Answer: Deep knowledge about the cases included in the study is important because it helps you interpret the results of the analysis. For instance, you might find a set of circumstances associated with a given outcome in most but not all cases with that outcome. Deep knowledge of the cases then allows you to go back and consider what about the cases that don't fit the pattern is different and maybe explain their deviance from the pattern.

Question: Is it possible to use fsQCA if you do not pre-select your factors for analysis?

Jodi's Answer: Yes you can. It just helps a lot if you know in advance what data to collect.

Marcus's Answer: Yes, you can work with existing data sets that were not collected with fsQCA in mind. That said, case selection and conceptualizing the conditions with set theory in mind makes the subsequent analysis easier and makes it easier to produce a high quality study.

Question: Is there any software that supports fsQCA?

Jodi's Answer: Yes it is on Charles Ragin's Web site from the University of Arizona: http://www.u.arizona.edu/~cragin/fsQCA/software.shtml.

Marcus's Answer: STATA and R also have QCA modules.

Question: In Marcus' example of PCMH  [Patient-Centered Medical Home] Level: what is the difference between leaving the actually level in (0, 1, 2, 3) and converting them to a number between 0 and 1? Essentially, wouldn't it accomplish the same thing? Is it just to make the math work?

Marcus's Answer: The NCQA [National Committee for Quality Assurance] PCMH levels are ordinal but not otherwise substantively related to performance. Using a membership function to assign fuzzy membership in the set "PCMH" requires the investigator to make a decision based on theoretical or substantive/empirical knowledge as to how meaningful the differences between the different NCQA levels are. Also, the math requires membership values between 0 and 1.

Question: Are there parallels to the concept of coverage and consistency in regression analysis?

Marcus's Answer: Consistency is somewhat analogous to the correlation constant r. It measures the strength of the association. Coverage is analogous to r2. It indicates how empirically important the set relationship is. However, unlike r and r2, consistency and coverage are not monotonically related. Relationships with high consistency and coverage have high correlations.

Question: Have you encountered objections from "traditional" methodologists, and if so, how did you address them?

Marcus's Answer: Yes, but mostly due to lack of familiarity. There is extensive literature contrasting and comparing traditional statistical methods with QCA methods in the sociology and political science literature. This should be helpful in health services research as well.

Question: If you want to learn this method, are you aware of any workshop or training opportunities?

Jodi's Answer: None that I know of. That is a problem.

Marcus's Answer: Ragin has, in the past, offered a Fall course on this at the University of Arizona. Now that he is at the University of California Irvine, I suspect he'll continue something like this. But you should check with him. The Web site www.compasss.org (yes, that's three S's) posts training opportunities and is a great source of info and updates on the method. Most of the training opportunities are in the social sciences but should be useful. Rubbing shoulders with organizational scientists, sociologists, and political scientists is illuminating. Health care is, after all, a social phenomenon.