Percy - Detailed Review
Coding Tools
Percy - Product Overview
Percy Overview
Percy, available at percy.io, is a visual testing and review platform that plays a crucial role in the coding tools and AI-driven product category. Here’s a brief overview of its primary function, target audience, and key features:
Primary Function
Percy is designed to automate visual testing for web and mobile applications. It captures screenshots of the user interface, compares them against established baselines, and highlights any visual changes or discrepancies. This process helps teams identify and address visual bugs before they reach users, ensuring a consistent and high-quality user experience.
Target Audience
Percy is primarily targeted at software development teams, including developers, designers, and QA engineers. It is particularly useful for organizations that need to ensure their UI components are visually consistent and free from regressions across various devices and browsers.
Key Features
Integration
Percy seamlessly integrates into existing CI/CD pipelines and works with popular tools like GitHub, GitLab, Jenkins, and more. This allows teams to automate visual testing as part of their regular development workflow.
Automated Testing
It captures UI snapshots, compares pixels with baselines, and detects visual bugs with every commit. This ensures that any visual changes are identified and reviewed promptly.
Review and Collaboration
Percy organizes visual changes, filters out noise, and updates teams with comments and notifications. This streamlines the review process and enhances collaboration between developers, designers, and QA teams.
Cross-Device Testing
Percy tests visuals on over 20,000 real devices, ensuring that the UI looks flawless across different browsers and devices. This includes DOM snapshotting and advanced parallelization to accelerate releases.
Efficiency and Confidence
By automating visual testing, Percy reduces the time and effort required to manually spot visual regressions, allowing teams to deploy code changes with greater confidence.
Overall, Percy is a valuable tool for any development team looking to maintain high visual standards and prevent unintended visual regressions in their applications.
Percy - User Interface and Experience
Percy Overview
Percy, a visual testing and review platform, is designed to make the process of ensuring the visual integrity of your applications as seamless and efficient as possible.
User Interface
The user interface of Percy is structured to be intuitive and user-friendly. Here are some key aspects:
Project Organization
Percy allows you to create multiple projects, each serving as an organizational structure for your visual tests. You can easily manage and configure settings for each project, including browsers, visibility settings, and automatic branch approvals.
Screenshot Rendering and Asset Discovery
Percy captures DOM snapshots and page assets, such as CSS and images, rather than accepting pre-captured images. This process is handled on a dedicated infrastructure, ensuring stable and scalable visual testing. The platform also includes features like freezing animations and handling dynamic data to make screenshots more stable.
Visual Review and Comparison
Percy enables you to compare screenshots against a baseline, highlighting any visual changes. This is presented in a visual diff format, similar to code diffs, making it easy to identify and review changes. The platform also supports visual review across different browsers and responsive screen sizes.
Integration with CI/CD Pipelines
Percy integrates seamlessly with your existing CI/CD pipelines, allowing you to run visual tests with every commit. It supports various automation frameworks like Cypress, Puppeteer, and Selenium, and can be integrated with GitHub, GitLab, and other version control systems.
Ease of Use
Percy is designed to be easy to use, even for teams without extensive experience in visual testing:
Simple Integration
Integrating Percy into your workflow requires minimal code changes. You can add visual testing to your test suite with just a few lines of code, making it accessible to developers and QA teams alike.
Automated Processes
Percy automates the capture and comparison of UI snapshots, detecting visual bugs and highlighting changes. This automation reduces the manual effort required for visual testing, allowing teams to focus on other aspects of development.
Clear Visual Feedback
The platform provides clear and actionable feedback through visual diffs and notifications. This helps teams quickly identify and address visual issues before they reach production.
Overall User Experience
The overall user experience with Percy is centered around efficiency and clarity:
Streamlined Workflow
Percy streamlines the visual testing process by organizing visual changes, filtering noise, and updating teams with comments and notifications. This ensures that visual bugs are caught early and addressed promptly, enhancing the overall quality of the application.
Confidence in Deployments
By automating visual testing and providing comprehensive feedback, Percy gives developers and QA teams more confidence when deploying changes to production. This reduces the risk of visual regressions and ensures a consistent user interface.
Cross-Platform Support
Percy supports testing across a wide range of devices and browsers, including real devices, which helps in ensuring that the application looks correct across different environments.
In summary, Percy’s user interface is designed to be intuitive, easy to use, and highly integrated with existing development workflows, making it an effective tool for automating visual testing and ensuring a high-quality user interface.
Percy - Key Features and Functionality
Percy: An AI-Driven Visual Testing Tool
Percy offers several key features that streamline and enhance the visual testing process for web and mobile applications. Here are the main features and how they work:Automatic Diff Matching
Percy’s Automatic Diff Matching feature groups screenshots with the same visual changes, making it easier to review builds. This allows users to approve or request changes for all matching changes with a single click, and expand the group to inspect individual screenshots. This feature simplifies the review process by consolidating similar changes, reducing the time spent on manual reviews.Diff Highlighter
The Diff Highlighter is a static bar that highlights the areas where changes are located in a new image. This tool helps identify differences quickly without the need to zoom in or enable the diff overlay, making visual comparisons more efficient.Configurable Diff Sensitivity
Users can adjust the diff sensitivity to differentiate between the base build and the head build on a project. This setting allows for five different sensitivity levels, ranging from Strict to Relaxed, depending on the project’s requirements. This flexibility ensures that the tool can be adapted to various testing needs.Percy Visual Engine
The Percy Visual Engine is a proprietary rendering engine that ensures accurate and consistent visual testing. It includes features like snapshot equilibration technology, which reduces false positives, and supports custom fonts and automatic freezing of animations. This engine helps maintain a pixel-perfect UI across different screen widths and responsive breakpoints.Integration with CI/CD and Testing Frameworks
Percy integrates seamlessly with Continuous Integration/Continuous Deployment (CI/CD) environments and various testing frameworks such as Appium, Java, Python, Webdriver IO, C#, and Ruby. This integration allows users to capture screenshots of the application’s pages in parallel every time a commit is run, eliminating the need to create functional test scripts from scratch for visual testing use cases.Smart Diff Technology
Percy uses smart diff technology to capture screenshots, differentiate them against the baseline, and underline visual changes. Only components and pages with changes are highlighted with diffs, making it easier to identify and review visual modifications. This technology ensures that users can have confidence in their UI across all screen widths and responsive breakpoints.Effortless Visual Reviews
Percy streamlines the visual review process by maintaining baselines, branch logic, and approval carry-forwards. Users can approve all screenshots in an entire build, approve groups of matching visual changes, or approve individual screenshots. Once all screenshots in a build are approved, the entire build is approved, keeping reviews in sync with workflows at all times.AI Integration
Percy leverages AI in several ways:- Intelligent Text Stabilization: AI helps stabilize text elements to ensure consistent rendering across different builds.
- Anti-aliasing Noise Handling: AI algorithms handle anti-aliasing noise, reducing false positives and improving the accuracy of visual comparisons.
- Page Shift Detection: AI detects and adjusts for page shifts, ensuring that visual changes are accurately identified and highlighted.
Public Projects and Transparency
For open-source projects, Percy provides the ability to run visual reviews transparently, giving contributors insight into the visual impact of any given code change. This feature enhances collaboration and transparency within development teams. By integrating these features, Percy simplifies and automates the visual testing process, allowing developers to ship software faster and with greater confidence in the visual integrity of their applications. Percy - Performance and Accuracy
Performance and Accuracy Evaluation of Percy
Visual Accuracy
Percy, unlike Applitools, struggles with distinguishing between dynamic and static content. This limitation leads to a higher rate of false positives, as Percy cannot accurately handle small, imperceptible rendering differences or dynamic content changes. This issue can result in unnecessary alerts and a decreased trust in the test results.Automated Maintenance
Percy lacks the ability to automatically group similar bugs together, which is a feature that Applitools offers. This means that Percy users have to manually review the same bugs repeatedly, slowing down the testing process in CI/CD pipelines and reducing confidence in the test results.Multi-Baseline Testing
For A/B testing, Percy is not capable of testing checkpoints against multiple baselines. This limitation forces tests to fail when encountering different variants, or it requires additional code to be written to handle these variations. In contrast, Applitools allows for multiple variants of a baseline, making it easier to test bold experiments.Cross Browser Testing
Percy’s cross-browser testing capabilities are limited compared to Applitools. Percy only supports testing on six different configurations (four web browsers and two mobile browsers), and these configurations are updated periodically. This leaves users vulnerable to bugs on the latest browsers. Applitools, on the other hand, offers a more comprehensive testing grid that can run tests up to 9x faster.General Limitations
While Percy is a useful tool for visual testing, its limitations highlight areas where it could improve. The inability to handle dynamic content, lack of automated bug grouping, and restricted cross-browser testing capabilities are significant drawbacks. These issues can lead to increased manual effort, slower testing cycles, and potentially missed bugs.Conclusion
In summary, while Percy serves its purpose in visual testing, it faces significant challenges in areas such as visual accuracy, automated maintenance, multi-baseline testing, and cross-browser testing. Addressing these limitations would be crucial for enhancing its performance and accuracy. Percy - Pricing and Plans
Percy Pricing Overview
Percy, a visual testing and review platform, offers a structured pricing structure with several plans to cater to different needs.
Free Plan
- This plan is free of charge and includes up to 5,000 screenshots per month.
- It supports unlimited users and unlimited projects.
- Features include cross-browser support, responsive visual regression testing, and a 30-day build history.
Professional Plan
- This plan costs $99 per month.
- It includes up to 25,000 screenshots per month.
- All features from the free plan are included, plus a 1-year build history.
- This plan also supports unlimited users.
Enterprise Plan
- For larger or more complex needs, Percy offers a custom Enterprise Plan.
- Pricing and specific features for this plan are available upon request from the BrowserStack sales team.
- This plan typically includes additional features such as single sign-on and a dedicated customer service manager.
Additional Considerations
- Screenshots usage beyond the included units in any plan is treated as overage and is charged at the overage rate.
- Percy’s usage is based on the number of screenshots rendered, with each screenshot being a rendering of a page or component in an individual browser and responsive width combination.
Conclusion
By choosing the appropriate plan, users can ensure they have the necessary resources for their visual regression testing needs without exceeding their budget.
Percy - Integration and Compatibility
Percy Overview
Percy, a visual testing and review platform, integrates seamlessly with a variety of tools and supports a broad range of platforms and devices, making it a versatile solution for developers.
Integration with Testing Frameworks
Percy can be integrated with numerous end-to-end testing frameworks, including but not limited to JestJS, TestNG, Nunit-C#, Mocha JS, WebdriverIO, Cucumber-TestNG, XUnit, SpecFlow, JUnit4, Serenity, Selenide, and JBehave. This integration allows you to run visual tests alongside your existing functional and behavioral testing setups.
For mobile applications, Percy’s extension, App Percy, supports frameworks like Appium, Java, Python, Webdriver IO, and C#, with Ruby support planned for the future. This ensures that you can manage your entire visual testing workflow for mobile apps from a single platform.
Cross-Browser Testing
Percy enables cross-browser visual testing, supporting the latest versions of modern browsers such as Chrome, Firefox, Edge, and Safari. You can configure browser versions using the BrowserStack SDK with Automate, and each build will display separate screenshots for each browser, highlighting visual differences specific to each browser.
Device Compatibility
For mobile devices, App Percy leverages BrowserStack’s real device cloud, offering over 20,000 real devices across 19 global data centers. This allows you to test on actual devices rather than emulators, ensuring higher accuracy. Supported devices include iPhones with iOS 15.x and Samsung S22 with Android 12.x.
CI/CD and Source Code Management
Percy integrates well with various Continuous Integration/Continuous Deployment (CI/CD) tools such as Jenkins, Circle CI, GitHub Actions, GitLab CI, Travis CI, Codeship, Semaphore, and Buildkite. It also supports source code management systems like GitHub, Bitbucket, Azure, and GitLab. This integration enables you to automate your visual testing within your existing DevOps workflows.
Visual Testing Automation
To initiate visual testing, you can use the Percy Snapshot feature, which is the simplest way to start your visual testing journey. The `@percy/cli` npm package provides the core SDK for common tasks like authentication, environment detection, and uploading screenshots to your Percy project. This allows you to capture screenshots automatically or at specific points during a test, and Percy will highlight any visual changes for review.
Custom Integrations
If your specific testing framework or technology is not listed, Percy provides the flexibility to build your own integration using the Percy CLI. This makes it possible to extend Percy’s capabilities to any new framework or tool you might be using.
Conclusion
In summary, Percy offers comprehensive integration options with various testing frameworks, cross-browser testing capabilities, and compatibility with a wide range of devices and CI/CD tools, making it a powerful tool for automating visual testing in your development workflow.
Percy - Customer Support and Resources
Customer Support
For immediate assistance, users can contact Percy’s Support team through a contact form available on their Support Center page. This allows users to submit tickets and receive help for any product-related queries or issues they might encounter.
Documentation and Guides
Percy provides comprehensive documentation that includes frequently asked questions, troubleshooting guides, and detailed instructions on various aspects of the platform. This documentation covers topics such as adjusting diff sensitivity, ignoring regions to avoid false positive visual diffs, base build selection with Git, comparing URLs, taking visual snapshots using the command line interface, and more.
Community and GitHub Resources
Percy has an active presence on GitHub, where users can find various repositories related to integrating Percy with different tools and frameworks. These repositories include visual testing with Protractor, Ember, Storybook, Selenium, Playwright, WebdriverIO, and Appium, among others. These resources can help users set up and customize their visual testing environments.
Tutorials and Getting Started Guides
For new users, Percy offers step-by-step tutorials and guides on how to get started with visual testing. For example, the DigitalOcean tutorial provides a detailed guide on setting up PercyScript for visual testing on a web app, including installing the necessary packages and writing the first visual tests.
Plans and Billing Support
Users can also find detailed information on plans and billing, including how to check screenshot usage, set up usage notifications, and manage their account settings. This ensures that users are well-informed about their usage and can manage their plans effectively.
Integration Guides
Percy provides guides on integrating their platform with other tools like GitHub, which helps users automate their visual testing as part of their CI/CD pipelines. These guides ensure seamless integration and efficient use of the platform.
By leveraging these resources, users can effectively use Percy for their visual testing needs and resolve any issues that may arise during the process.
Percy - Pros and Cons
Advantages
Automated Visual Testing
Percy automates the process of visual testing by capturing screenshots, comparing them against a baseline, and highlighting visual changes with every commit. This ensures that visual bugs are identified before they reach users.
Integration with CI/CD Pipelines
Percy seamlessly integrates into existing CI/CD pipelines, allowing for automated visual testing as part of the development workflow. It supports various automation frameworks and tools like Github, Gitlab, Jenkins, and more.
Efficient Review Process
The platform streamlines the review process by grouping screenshots with the same visual changes, allowing for quick approval or request for changes. Features like the diff highlighter and configurable diff sensitivity make it easier to identify and review changes.
Cross-Device Testing
Percy tests visual changes across over 20,000 real devices, ensuring that the UI looks correct on various devices and screen sizes. This includes testing behind logins or firewalls, which is managed through automation scripts.
Enhanced Team Collaboration
Percy supports issue and pull request management in platforms like Github, Bitbucket, and Gitlab, making it easier for teams to review and manage visual changes during the development process.
Resource Efficiency
Percy handles the rendering and generation of screenshot diffs on its own infrastructure, which means it does not waste the user’s computing resources.
Disadvantages
Dependence on Automation
While Percy automates many aspects of visual testing, it still requires setup and integration into the existing workflow. This can be a bit of an initial overhead.
Potential for False Positives
The tool may sometimes highlight changes that are not significant or are due to minor variations in rendering, which can lead to unnecessary reviews.
Configuration Requirements
Users need to adjust settings like diff sensitivity to suit their project’s requirements, which can take some time to get right.
Limited Context for Non-Visual Issues
Percy is specifically designed for visual testing and does not catch non-visual issues such as functional or performance problems. It needs to be used in conjunction with other testing tools.
Overall, Percy is a valuable tool for ensuring the visual integrity of web applications, but it should be part of a broader testing strategy that includes other types of testing.
Percy - Comparison with Competitors
When Comparing Percy with Competitors
When comparing Percy, a visual testing tool, with its competitors in the AI-driven testing category, several key differences and unique features become apparent.
Visual Accuracy and Dynamic Content
Percy, despite its recent improvements with the Percy Visual Engine, still faces challenges in testing dynamic content. The new engine uses computer vision and machine learning to reduce false positives and highlight exact pixel changes, which is an improvement over its previous version. However, Applitools stands out in this area by leveraging AI-powered Match Levels, which enable it to test dynamic content like ads, news, or dashboards with high accuracy and minimal false positives. Applitools can handle dynamic interfaces more effectively than Percy, making it a stronger choice for applications with personalized or frequently changing content.
Test Maintenance and Automation
Percy’s ability to automate test maintenance is limited compared to Applitools. Applitools can automatically group similar bugs together and update new tests across all relevant components with a single click, significantly reducing manual review time. In contrast, Percy requires more manual intervention, leading to slower testing cycles and reduced confidence in test results.
Multi-Baseline Testing and Cross Browser Testing
Applitools supports multi-baseline testing, allowing frontend teams to create tests for A/B experiments easily. This feature is lacking in Percy, which would require additional code to handle variant testing, leading to test failures each time a variant is encountered. Applitools also excels in cross-browser testing with a next-gen visual testing grid that runs tests up to 9x faster than other grids. It supports a broader range of browsers and devices compared to Percy, which is limited to 6 configurations (4 web browsers and 2 mobile browsers) that are updated periodically.
Root Cause Analysis and Deployment
Applitools provides detailed root cause analysis, pinpointing the exact CSS or HTML changes that caused UI bugs. Percy lacks this feature, requiring hours of manual debugging to identify the root cause of issues. In terms of deployment, Applitools offers more flexibility by allowing deployment on a public cloud, private cloud, or on-premises, which is crucial for large enterprises with strict security requirements. Percy, on the other hand, is only available as a public SaaS application, which may pose security concerns for some organizations.
Alternatives
For those looking for alternatives to Percy, here are some options:
Applitools
As discussed, Applitools is a strong competitor with advanced AI-powered visual testing capabilities, better handling of dynamic content, automated test maintenance, and comprehensive cross-browser testing.
Testim
Testim is another alternative that offers AI-powered test automation. It supports end-to-end testing and provides features like self-healing tests and AI-driven impact analysis, which can reduce the initial implementation time and maintenance efforts.
OpKey
OpKey is a test automation tool that supports various technologies and offers a Core Automation Engine for building end-to-end test cases. It includes smart tools for mobile app testing and a self-healing engine to reduce maintenance time.
Conclusion
In summary, while Percy has made significant improvements with its Visual Engine, it still lags behind competitors like Applitools in terms of handling dynamic content, automated test maintenance, and comprehensive cross-browser testing. Considering these factors, Applitools and other alternatives like Testim and OpKey may offer more comprehensive solutions for AI-driven visual testing needs.
Percy - Frequently Asked Questions
Frequently Asked Questions about Percy
How do I get started with Percy?
To get started with Percy, you need to install one of its SDKs into your project. This process is similar to setting up a typical testing framework like Mocha or Jest. After installation, you create a script to capture DOM snapshots of your web application, which are then uploaded to Percy for rendering and comparison against baselines.
What is the approval workflow in Percy?
The approval workflow in Percy involves several steps. You create a new feature branch, make and commit changes, and then create a pull request via your GitHub dashboard. Percy tests are run as part of this process, and team members can review and approve the visual changes in the Percy dashboard. Once approved, the feature branch is merged, and a new baseline is created on the master branch.
How does Percy integrate with CI/CD pipelines?
Percy integrates seamlessly with various CI/CD tools such as GitHub, GitLab, and Jenkins. This integration allows you to automate visual testing as part of your continuous integration pipeline. Percy captures UI snapshots, compares them against baselines, and detects visual bugs with every commit, ensuring that visual changes are reviewed and approved before deployment.
What devices and browsers does Percy support?
Percy supports testing across a wide range of devices and browsers. It can test on over 20,000 real devices, including mobile and desktop browsers, ensuring that your UI looks correct across different screen sizes and devices.
How does Percy handle dynamic data and responsive design?
Percy supports dynamic data handling and responsive design testing. It captures screenshots at various responsive breakpoints and highlights visual changes, ensuring that your UI looks correct across all screen widths. Additionally, Percy’s proprietary snapshot equilibration technology reduces false positives, and it can maintain custom fonts and freeze animations automatically.
Can I use Percy with existing testing frameworks?
Yes, Percy is designed to work with existing testing frameworks. It provides SDKs that integrate with frameworks like Cypress, Jest, and others, making it versatile for different development environments. This integration allows you to add visual testing to your existing test suite with minimal code changes.
What is the free plan offered by Percy?
Percy offers a free plan that supports unlimited team members, 5,000 snapshots per month (with a one-month history), and unlimited projects. This plan is suitable for small projects or teams looking to get started with visual testing.
How does Percy handle visual bug detection and reporting?
Percy captures UI snapshots and compares them against baselines to detect visual bugs. It highlights visual differences and organizes changes for easy review. Team members can comment and receive notifications about visual changes, and Percy updates the status of pull requests to reflect whether changes have been approved or need further review.
Can I automate the visual testing process with Percy?
Yes, Percy is designed to be part of automated testing processes. You can automate the capture and comparison of visual snapshots using Continuous Integration platforms. This automation ensures that visual tests are run with every commit or merge, providing continuous visual coverage.
How does Percy support teamwork and collaboration?
Percy streamlines teamwork by allowing multiple team members to review and approve visual changes. It provides features like grouped snapshots, noise filtering for swift reviews, and auto-status updates in pull requests. This collaborative approach ensures that only intended changes are deployed to production.
What kind of support and resources does Percy offer?
Percy offers various support resources, including documentation, FAQs, and integration guides. For more specific issues, you can refer to the support section on the BrowserStack website, which includes troubleshooting tips and answers to frequently asked questions.
Percy - Conclusion and Recommendation
Final Assessment of Percy
Percy is a highly effective visual testing tool that integrates seamlessly into the development workflows of software teams, particularly those focused on web applications. Here’s a detailed assessment of who would benefit most from using Percy and an overall recommendation.Key Benefits
- Automated Visual Testing: Percy automates the visual QA process, allowing teams to catch visual bugs and review design changes efficiently. It integrates with popular version control systems like GitHub, GitLab, and Bitbucket, making it easy to add visual reviews to pull requests and automate visual testing for every commit.
- Visual Diffs and Consistency: Percy provides simple and fast visual diffs at scale, handling consistent screenshot rendering and visual diffing without impacting the performance of the team’s tests. This ensures that every pixel deployed to customers is correct, maintaining the visual integrity of the UI across all screen widths and devices.
- Cross Browser and Device Testing: Percy supports cross browser and device testing, ensuring that web applications are compatible across various browsers and devices, including desktop and mobile browsers.
- Efficiency and Collaboration: By automating visual testing, Percy improves efficiency by reducing the time and effort required to manually spot visual regressions. It also enhances collaboration between developers, designers, and QA teams by providing a clear visual representation of changes.
Who Would Benefit Most
- Development Teams: Teams involved in web application development, especially those with a strong focus on UI and UX, would greatly benefit from Percy. It helps in identifying even the slightest changes in the UI and provides effective reviews and components with changes, ensuring a consistent user experience.
- QA Teams: Quality Assurance teams can leverage Percy to automate the visual QA process, reducing manual effort and improving the accuracy of visual testing.
- Designers: Designers can use Percy to review design changes and ensure that the visual aspects of the application align with the intended design specifications.