The problem

Right now any POST request containing a file payload is failling on my end and ends up crashing the expose client due to memory limitations.

The cause

This is happening because all requests seem to go through a logging process that require parsing the whole request body.

On multipart/form-data POST content this is not a very good idea since the body is likely to contain a big amount of data which the parser won't be able to handle.

Proposed solution

Avoid logging POST upload requests.

This fixes the problem completely on my end and I'm now able to perform upload requests without issues while using expose.

If logging these requests is a necesity then another approach must be used as to only grab part of request and avoiding parsing the whole body and it is being done right now.

I hope this helps and ends up being useful for you.

Related tickets

• https://github.com/beyondcode/expose/issues/64

Hey,

You've created an amazing tool and I'm going to run on my own server (with Docker). I see that a change on entrypoint in #149 causes a bug with CMD in Docker see in #165.

I purpose to create a custom endpoint to keep behavior for run default cmd and also a custom command if you want in your docker run.

I'm waiting your advise especially for entrypoint sh file that I've put on root but maybe you want to create a special folder for "Docker stuff".

In the current version, the host header is set incorrectly, which causes applications to incorrectly determine the correct host.

This PR fixes the error and sends the application the correct current header.

Also, two new X- headers are introduced to recognize what the exposed host + proto was, so that the proxy browser can determines the correct hostname, port and schema.

• X-Expose-Host: <hostname>:<port>
• X-Expose-Proto: <schema>

Furthermore a DSN support was introduced. So that also exposed HTTPS connections can be resolved.

Other changes in this pr:

• Changed the X-Expose-Request-ID to sha1 so it contains a hex string with a more entrophy uniqueid.

--

I created this PR to share insights. This requires a major release and i think, there are few more places that should be reworked before a major release.

This PR adds a built-in fileserver to expose.

Possible commands are:

# Share the current working directory
expose share-files

# Share a specific directory
expose share-files ~/Documents

# Share a specific directory, but filter the shared files using a regex/glob
expose share-files ~/Documents --name="*.jpg"

As this starts an internal HTTP server, it can be combined with all existing expose options, like specifying the subdomain, or adding basic authentication.

This Pull Request allows a user to configure the option ssl to enforce use of SSL or no SSL within the tunnel It was made in such a way that the default of null will retain the current logic that expose uses which is port === 443

It is useful if you want to host expose on something other than port 443 while still using SSL. Something else that could be added is warning a user if they use 'ssl' => false, 'port' => 443 as most browsers will reject this

My git skills failed me on #89 so I recreated the commits.

My little 2 cnts to project.

Commands factorization to avoid duplicate code adding some helper methods on a new abstract command class.

Updated methods based on Single-responsibility principle, less lines on single methods is always welcome :)

Added section on detecting Expose as the active proxy server. This can be useful when taking custom action based on the kind of proxy server being used (such as overriding non-standard routing logic, auto-selecting proxy-specific databases, etc)

Ref: https://github.com/beyondcode/expose/blob/d98eabe36e9c19c42f3b8f44601a2a457cb3af26/app/Server/Http/Controllers/TunnelMessageController.php#L106-L124

This PR. allow you to display payload data.

I think this is more rigorous

see issue: https://github.com/beyondcode/laravel-websockets/issues/362

As the Event Creator is mostly used for debugging I feel that logging should be enabled by default.

Closes #321

I make the user table and user reference key set in config file, it will be more flexible for those not using User table as their default authenticate model.

For my case, I am using Customer table as the model.

This fixes compatibility with for example laravel-mail-css-inliner which registeres a Swift_Mailer plugin (and potentially other packages that register Swift_Mailer plugins too) by re-using the Swift_Mailer instance Laravel creates on a Mailer.

Fixes #5

There is a bug in Laravel (https://github.com/laravel/framework/issues/17283) that prevents Mailable with callbacks to be serialized for queueing.

So when a Mailable implements ShouldQueue we don't add headers.

Also changed the check from MailableContract to Mailable since the applyDebugHeaders function expects a Mailable (that has a view property).

It will allow it to install on php 8.x, 9.x etc. but ensure that at least 7.1 is available.

See the above on https://getcomposer.org/doc/04-schema.md#require

Not sure if it was done thoroughly. But needed it for a project and it worked perfectly. Feel free to change anything.

Just spotted this and thought that it's a quick one to fix!

Some projects may have too many models to draw, so I add this feature about focusing some models I want to see.

Hello, This PR basically adds SES support to Laravel Mailbox. It is overcoming few shortcomings from Amazon SNS.

1. The POST request from SNS has content-type as text instead of JSON.
2. SNS posts a request first without the Authorization header, and expects a response of 401 Unauthorized and Realm.

Appreciate your review and comments.

Hello @mpociot, as promised, here comes the big change PR. Take a look and feel free to comment. Independently of the outcome, I will continue to support my version since I need it anyways. I understand if this is too much of a change to accept, and that is fine as well. Thanks in advance.

Change log

Concept

The main entity is a MailboxGroup. What it enables you to do is to have multiple Mailboxes, each with its own set of patterns and actions to execute.

Refactor has been done so that Mailbox provides a fluent API to define same set of features as before with some nice additions.

Example:

$mailbox = new Mailbox();

$mailbox
    ->from('{user}@gmail.com')
    ->to('{user2}@gmail.com')
    ->subject('Subject to match')
    ->where('user', '.*')
    ->where('user2', '.*')
    ->action(function (InboundEmail $email) {
        Log::info("Mail received");
    })
    ->matchEither()
    ->priority(10);

The concept now is that each mailbox has a single or multiple rules it can match, however it can have only one action to execute. Reason being to enable matching all or either of the rules imposed.

By default, all patterns need to be matched in order for action to be executed. So if you provide from and to, they both need to match. This can be changed by including a matchEither() function which will transform this logic to matching at least one.

Due to the fact that multiple mailboxes can exist now, MailboxGroup is responsible for running them. Executing MailboxGroup::run($email) will run all the mailboxes, but will before order them by given priority(). Mailbox has a default priority of 0. The bigger the number, the sooner it is executed.

Group example:

$group = new MailboxGroup(); 

$group
    ->add($mailbox)
    ->add($mailbox2)
    ->add($mailbox3) 
    ->fallback(function (InboundEmail $email) {
        Log::info("Fell back");
    })
    ->continuousMatching()
    ->run($email);

You are responsible for adding mailboxes to the group (be it in the service provider or the controller itself). Fallback is pulled out to MailboxGroup for logical reasons. If none of the mailboxes match, fallback will be executed. A new feature here is that mailboxes will, by default, stop at first match. Meaning, if out of 5 mailboxes, second one matches the incoming mail, execution will stop. This can be overridden including the continuousMatching() function which will run through all mailboxes. So if match is found in 3/5 mailboxes, those 3 callbacks will be executed.

Dependencies

PHP version has been bumped to 7.4. Breaking changes are introduced, 7.3 is no longer supported, and in the next few days I will complete all type hints and return types, together with strict modes included.

Support for 6.x Laravel has been dropped. composer install requires a ton of memory with it, breaking most of the times if you don't give more memory to it. This is also the reason Scrutinizer checks break.

Drivers

I have removed drivers all together. Reason being that they were (at least for me) ambiguous, and their only purpose was exposing routes behind the scenes. Current setup also doesn't limit you in using a single driver.

I like my code to be explicit, so currently in order to set up the package you need to:

1. Set up endpoint manually (I like single action controllers for these purposes, but not mandatory):

Route::post('receive', ReceiveEmailController::class);

2. Add a MailboxRequest form request validation, or extend it if needed.

public function __invoke(MailboxRequest $request)
{
    ...
}

3. Define mailboxes and run them:

public function __invoke(MailboxRequest $request)
{
    $mailbox = (new Mailbox())->from(...)->action(...);

    MailboxGroup::add($mailbox);
    MailboxGroup::run($request->email());
}

4. Alternatively, you can define mailboxes in your service provider boot method, leaving the controller clean:

// App service provider
public function boot()
{
    $mailbox = (new Mailbox())->from(...)->action(...);

    MailboxGroup::add($mailbox);
}

// Controller
public function __invoke(MailboxRequest $request)
{
    MailboxGroup::run($request->email());
}

5. Alternatively, option I prefer is to pull out mailbox definition to a separate class and include it within the controller so that it doesn't overburden the rest of API calls.

public function __invoke(MailboxRequest $request)
{
    // Factory leading to defining mailboxes, and ultimately running them.
    MailboxService::receiveEmail($request->email());
}

6.. Controllers being used as driver callbacks are still there for your convenience, so you can point the routes to different variants of those:

Route::post('receive/mailgun', MailgunController::class);
Route::post('receive/mailcare', MailCareController::class);
Route::post('receive/postmark', PostmarkController::class);
Route::post('receive/sendgrid', SendGridController::class);

Dev naming

Some concepts, variables etc. have been renamed to be more intuitive.

Tests

Tests which made sense were modified to pass, other were deleted as things like Route and RouteCollection no longer exist. I am willing to support new features with tests.

Readme

I think readme is a tad outdated and unnecessarily long. I will probably rewrite it.

Hello,

I've modified the code to provide truly extensible code with configuration which I deem is simpler than overriding complete classes and re-binding them.

Some of the changes are also due to what I consider misleading nomenclature, but you will find those changes are not so drastic.

I've also added readme for extending.

Keep up the good work, and hit me back if something else is needed.

Regards

Hi,

I have added two new features.

1) Ability to exclude a table from the export:

Sometimes you might not want to include all tables in the export. You can achieve this with:

return [
    'default' => DumpSchema::define()
                    ->allTables()
                    ->exclude('password_resets')
                    ->exclude('migrations');
];

2) Ability to create INSERTs with multiple rows

When you have a table with many rows (1000+) creating INSERT statements for each row results in a very slow import process. For these cases it is better to create INSERT statements with multiple rows.

You can achieved this with ->outputInChunksOf($n).

return [
    'default' => DumpSchema::define()
        ->allTables(),
        ->table('users', function($table) { 
                return $table->outputInChunksOf(25); 
            });
];

Tests

• I have fixed the existing tests (they used artisan db:dump instead of db:masked-dump)
• I have added a test for exclude()
• I have added a test for outputInChunksOf()

Documentation

I have added two sections describing the features in the Readme.md file.

Liebe Grüße Alex

PS: It's my first pull request ever for an open source project ... so I hope I didn't miss anything :) PPS: Thank you for all the great work and stuff you are providing!

This was my original intention for forking the repo, but when I tried to run the tests I got the errors, so I made this PR https://github.com/beyondcode/laravel-masked-db-dump/pull/5

Now I've successfully run the tests on PHP 8 and everything seems fine.

Currently if you try to run the tests you get this error:

BeyondCode\LaravelMaskedDumper\Tests\DumperTest::it_can_dump_certain_tables_as_schema_only
Symfony\Component\Console\Exception\CommandNotFoundException: The command "db:dump" does not exist.

Because the actual command has been renamed from db:dump to db:masked-dump but the tests weren't updated.

After fixing that if you run the tests you get this one:

BeyondCode\LaravelMaskedDumper\Tests\DumperTest::it_can_replace_columns_with_faker_values
ArgumentCountError: Too few arguments to function BeyondCode\LaravelMaskedDumper\Tests\DumperTest::BeyondCode\LaravelMaskedDumper\Tests\{closure}(), 1 passed and exactly 2 expected

Because in it_can_replace_columns_with_faker_values we're receiving a $faker as dependency but we're never passing it in DumpSchema.php, and if we add that we become green.

Without this change, the documentation example works incorrectly: dumps the DB but doesn't use the correct mask character.

return [
    'default' => DumpSchema::define()
        ->table('users', function ($table) {
            $table->mask('password', '-');
        })
];

Hello @mpociot

Thanks for this package, Smart and neat! .. Used it in one project where it allowed me to fix N+1 issues in bulk throughout the code base, a real time saver. Thank you!

Without a breaking change, This PR includes the following:

- Basic code refactor

Targeting src/QueryDetector.php, Using Concerns traits, Return early if statements.

Commits:

5f2dfd686b5da2691e174e20df488abe5b50b005 Refactor to Return Early if statements 033258340db5d3eb4d4170a8ea1a38405d899114 Refactor to HasContext Concern 11bc17b6710ade8b186643c879fd0a92f8a2f41c Refactor to InteractsWithSourceFiles Concern 946d1ea3ad94d8ee911d1cd9cfce0acb71e57e31 Refactor to Bootable Concern

- Adaptability to multiple-requests contexts

Possibly closes: #70 Possibly closes: #71

Problem:

What I mean by multiple requests contexts, when we do multiple requests/responses lifecycles within one Laravel container app() instance.

Commonly this happens in Feature/Integration tests. Where we might do multiple requests inside one test.

To even simplify the problem, We already have in our unit tests this passing test:

    /** @test */
    public function it_does_not_fire_an_event_if_there_is_no_n1_query()
    {
        Event::fake();

        Route::get('/', function (){
            $authors = Author::with('profile')->get();

            foreach ($authors as $author) {
                $author->profile;
            }
        });

        $this->get('/');

        Event::assertNotDispatched(QueryDetected::class);
    }

If we just duplicated the visit $this->get('/'); $this->get('/');, it will fail and it will claim we are having N+1 issue.

Solution:

This was solved by adding a $context variable to be part of the calculated query key. So the key now includes the context variable, and we will automatically register a new random string context after each request.

The key will be (In src/QueryDetector.php) :

$key = md5($this->context . $query->sql . $model . $relationName . $sources[0]->name . $sources[0]->line);

And in the middleware we will register a new context after each request. (In src/QueryDetectorMiddleware.php) :

        /** @var \Illuminate\Http\Response $response */
        $response = $next($request);

        $this->detector->newContext();

        // Modify the response to add the Debugbar
        $this->detector->output($request, $response);

Now, a userland code for isolating queries in dispatched jobs might look something like this:

        Event::listen(function (JobProcessing $jobProcessingEvent) {
            app('querydetector')->newContext();
        });

Commits: a691bc2f801cacf530c213e8d9222808d1bac900 Always clear context after requests 7e370ddbb3fc0bf2d38dad876f0e6fc33becd8d8 Add Contexts Feature

- Other Bug Fixes and Enhancements

While working on this context feature, I had to solve some older issues.

1. Bug: Booting multiple times.

The code in the middleware before executing the request:

$this->detector->boot();

Now it's

$this->detector->bootIfNotBooted();

Since booting multiple times means registering multiple database listeners DB::listen since it's in the boot function.

Commit: 0f692b58c066b52fd308a37dac94e688246c2198 Safely boot the service in controllable and manageable way

2. Dispatching multiple events

The code in BeyondCode\QueryDetector\QueryDetector::output($request, $response) function, which get called from the middleware, will call the function getDetectedQueries multiple times.

This function, BeyondCode\QueryDetector\QueryDetector::getDetectedQueries, encapsulates dispatching the event QueryDetected.

The solution was to call getDetectedQueries once and pass it's results around to the needed methods.

I considered extracting the event dispatch logic from getDetectedQueries, to prevent a "getSomething" method from performing any actions silently, but then I thought that should be for further development in a separate PR.

Commit: a3ca8f2de87540895dd7792c52adb7e9b5b48f41 Fix dispatching multiple events

Integration with ajax requests to show the alert when an N+1 is detected

I could make it work, that the app is opened in Windows. The only thing that doesn't work, for now, is that the variables are not populated.

I hope my pull request helps to bring this feature as fast as possible to Windows... because it's awesome!!!

It works well.

Also fixed that editor config was not loading properly because of a typo.

• [x] Update composer.json to reflect that php8 is supported.
• [x] Update travis ci to also test against php8.

Windows can't handle TTY, so this checks the OS before setting the flag. Allows the user to configure the timeout, because 60 seconds is annoying if you're setting a lot of variables. Also allows parameters to be passed into the editor call, as Visual Studio Code, for example, requires a -w flag to block the terminal until the user is done editing.

This resolves issue #37 and #15. It coincidentally duplicates the effort in #38, since it's impossible to test without that fix. Kudos to @b0ric for the tty check conditions in #15.

Co-Authored-By: andrewkurzweil [email protected]

Hi,

I was having trouble upgrading to PHP 8 and updated the composer.json file.

Thanks!

-Rob

allow php8

I've added a slimmed down implementation of laravel-env-scanner to count undefined environmental variables used in config/ and app/ directories by env() & getenv() helpers.

When an in-memory caching is used in a shared hosting environment (e.g. because there's an additional staging system on the same server) and no cache prefix is set and Laravel sites have the same app name configured, Laravel will fall back to a default cache key prefix which leads to shared cache key access across sites.

I'm sorry, I have no idea why the language files are completely rewritten in the commit. Of course I just added the relevant lines and that's also what git diff indicated.

Fix issue with phpredis driver in the RedisCanBeAccessed checker

Related issue #68

After #39 I have been thinking about structuring the tests. In this PR are new test, structure changes and small changes in checks to prevent failures.

What I have done:

• Move the test to corresponding directories, in fact move all the test for checks to tests/Checks
• Ignore the phpunit.xml in .gitignore, there is already a phpunit.xml.dist and the phpunit.xml is only to overwrite that file so I suspect it can be safely ignored.
• Edit the DirectoriesHaveCorrectPermissions, ExampleEnvironmentVariablesAreSet, ExampleEnvironmentVariablesAreUpToDate, LocalesAreInstalled and ServersArePingable check so you don't get an error if you call the message function before the check function on that classes. (Normally that shouldn't happen but it isn't forced so I think it is good to prevent any error)
• Change a mock result in StorageDirectoryIsLinkedTest to match the real result.
• Add tests for the next checks:
  • ComposerWithDevDependenciesIsUpToDate
  • ComposerWithoutDevDependenciesIsUpToDate
  • ConfigurationIsCached
  • ConfigurationIsNotCached
  • DebugModeIsNotEnabled
  • MaintenanceModeNotEnabled
  • RoutesAreCached
  • RoutesAreNotCached
• Add test for the message and name functions on already existing tests
• Add @group annotations to the tess

Some questions I have:

• It is possible to use a trait in the test for the message and name function test of the most checks. I didn't do it to make it clear and readable. Is that good or is the use of a trait wanted?
• Is it wanted that I add a message to the assertInternalType calls for the message and name checks? Now I just be sure that it is working without any problem.
• I added the group checks to the tests for the checks and also groups for connected tests (Like @group routes ) Is this wanted or not? With the current size of the project I wasn't very sure if it wanted to add.
• Currently everything is in one PR. Is it wanted to split it? (Like PRs for the changes on the checks, a PR for the structure changes and PRs for the new tests?) And if wanted I can create a new one with more clear commits because I understand some of them aren't very clear.

Todo:

• Add a test for the PhpExtensionsAreInstalled check, this is the only check without tests so I should like to also add that one.

I should really like to get some feedback on the work that is done. Is something missing? Something done that isn't wanted? Do you see something strange in the code?

This package is awesome! Thanks so much.

I hit a snag with Horizon, which requires the ext-pcntl and ext-posix php extensions. My dev environment is Windows and my production environment is Linux. These extensions are unfortunately not available for Windows. The solution I found was to add the following to composer.json

  "config": {
    "platform":{
      "ext-pcntl": "7.1",
      "ext-posix": "1.0"
    }
  },

It works great. However, after installing laravel-self-diagnosis and running php artisan self-diagnosis the missing extensions came back to haunt me and fail the laravel-self-diagnosis required PHP extensions check.

This PR tells laravel-self-diagnosis to skip over any extensions listed in composer.json under config['platform']

You can toggle the setting in the config file under checks.ignore_composer_config_platform_extensions

I went a bit deep trying to find out how to add a message (ie an entry without a duration) to the server timing output - as it turns out, there is an example in the docs but it has somehow been pushed a section down into the 'Publishing configuration file' setting.

By that point, I'd forked the repo and created an addMessage utility.

Updated the config file to look for a SERVER_TIMING_ENABLED variable in the .env file, so that it can be enabled/disabled per environment.

This changes the CI Runner from travis CI to Github Actions.

TODO

• [x] Add PHP 8 from #17 as soon as its merged

This update allows for control of it's enable state. I think this should be something included out of the box, I know its a simple change but its something that I think it's helpful as lots of things are controlled via ENV vars in the end.

I have it reading the ENV Value and checking if its string(true), if so then true otherwise false. If no value is passed in then it defaults to true anyway so its the same result as if this was not added in.

Argument 1 passed to App\Http\Controllers\Auth\ForgotPasswordController::sendResetLinkResponse() must be an instance of Illuminate\Http\Request, string given, called

Fix the request argument does not send to sendResetLinkResponse

This only fixes the dump on page.

Not sure if this causes any side effects as there weren't any tests to run. I am using windows 10 pro wsl (ubuntu 18.04lts) with valet for linux and occasionally the laravel dump stops working - both(the cli and on page).

The only thing that helps it work again for a while is restarting the OS.

restarting php7.4-fpm, nginx, valet, clearing composer dump or clearing cache... Nothing works.

As Symfony 5.0 just came out we could update this packages as well.

There aren't really tests in the package, so let me know if stuff broke. I'll gladly fix it.

SymfonyHackday