Eventstore: Server doesn't start at default HTTP ports
Describe the bug
The server stopped starting when the default ports are used for whatever reason. The ports are NOT in use by another program. I've tried running the server as admin as well.
To Reproduce
Steps to reproduce the behavior:
Just starting the server:
.\EventStore.ClusterNode.exe
Expected behavior
Server should start at the default ports.
Actual behavior
Server doesn't start with the default ports. :(
Config/Logs/Screenshots
C:\webstack\event-store-5
位 .\EventStore.ClusterNode.exe --ext-http-port 2113 --int-http-port 2112
[23028,01,23:11:45.790]
"ES VERSION:" "5.0.4.0" ("oss-v5.0.4"/"aface1f1b14309a075a070051c2e8f35a67e824e", "Fri, 1 Nov 2019 14:15:31 +0100")
[23028,01,23:11:45.820] "OS:" Windows (Microsoft Windows NT 6.2.9200.0)
[23028,01,23:11:45.820] "RUNTIME:" ".NET 4.0.30319.42000" (64-bit)
[23028,01,23:11:45.820] "GC:" "3 GENERATIONS"
[23028,01,23:11:45.820] "LOGS:" "C:\webstack\event-store-5\logs"
MODIFIED OPTIONS:
EXT HTTP PORT: 2113 (Command Line)
INT HTTP PORT: 2112 (Command Line)
DEFAULT OPTIONS:
HELP: False (<DEFAULT>)
VERSION: False (<DEFAULT>)
LOG: C:\webstack\event-store-5\logs (<DEFAULT>)
CONFIG: <empty> (<DEFAULT>)
DEFINES: <empty> (<DEFAULT>)
WHAT IF: False (<DEFAULT>)
START STANDARD PROJECTIONS: False (<DEFAULT>)
DISABLE HTTP CACHING: False (<DEFAULT>)
MONO MIN THREADPOOL SIZE: 10 (<DEFAULT>)
INT IP: 127.0.0.1 (<DEFAULT>)
EXT IP: 127.0.0.1 (<DEFAULT>)
INT TCP PORT: 1112 (<DEFAULT>)
INT SECURE TCP PORT: 0 (<DEFAULT>)
EXT TCP PORT: 1113 (<DEFAULT>)
EXT SECURE TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
EXT SECURE TCP PORT: 0 (<DEFAULT>)
EXT IP ADVERTISE AS: <empty> (<DEFAULT>)
EXT TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
EXT HTTP PORT ADVERTISE AS: 0 (<DEFAULT>)
INT IP ADVERTISE AS: <empty> (<DEFAULT>)
INT SECURE TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
INT TCP PORT ADVERTISE AS: 0 (<DEFAULT>)
INT HTTP PORT ADVERTISE AS: 0 (<DEFAULT>)
INT TCP HEARTBEAT TIMEOUT: 700 (<DEFAULT>)
EXT TCP HEARTBEAT TIMEOUT: 1000 (<DEFAULT>)
INT TCP HEARTBEAT INTERVAL: 700 (<DEFAULT>)
EXT TCP HEARTBEAT INTERVAL: 2000 (<DEFAULT>)
GOSSIP ON SINGLE NODE: False (<DEFAULT>)
CONNECTION PENDING SEND BYTES THRESHOLD: 10485760 (<DEFAULT>)
CONNECTION QUEUE SIZE THRESHOLD: 50000 (<DEFAULT>)
FORCE: False (<DEFAULT>)
CLUSTER SIZE: 1 (<DEFAULT>)
NODE PRIORITY: 0 (<DEFAULT>)
MIN FLUSH DELAY MS: 2 (<DEFAULT>)
COMMIT COUNT: -1 (<DEFAULT>)
PREPARE COUNT: -1 (<DEFAULT>)
ADMIN ON EXT: True (<DEFAULT>)
STATS ON EXT: True (<DEFAULT>)
GOSSIP ON EXT: True (<DEFAULT>)
DISABLE SCAVENGE MERGING: False (<DEFAULT>)
SCAVENGE HISTORY MAX AGE: 30 (<DEFAULT>)
DISCOVER VIA DNS: True (<DEFAULT>)
CLUSTER DNS: fake.dns (<DEFAULT>)
CLUSTER GOSSIP PORT: 30777 (<DEFAULT>)
GOSSIP SEED: <empty> (<DEFAULT>)
STATS PERIOD SEC: 30 (<DEFAULT>)
CACHED CHUNKS: -1 (<DEFAULT>)
READER THREADS COUNT: 4 (<DEFAULT>)
CHUNKS CACHE SIZE: 536871424 (<DEFAULT>)
MAX MEM TABLE SIZE: 1000000 (<DEFAULT>)
HASH COLLISION READ LIMIT: 100 (<DEFAULT>)
DB: C:\webstack\event-store-5\data (<DEFAULT>)
INDEX: <empty> (<DEFAULT>)
MEM DB: False (<DEFAULT>)
SKIP DB VERIFY: False (<DEFAULT>)
WRITE THROUGH: False (<DEFAULT>)
UNBUFFERED: False (<DEFAULT>)
CHUNK INITIAL READER COUNT: 5 (<DEFAULT>)
RUN PROJECTIONS: None (<DEFAULT>)
PROJECTION THREADS: 3 (<DEFAULT>)
WORKER THREADS: 5 (<DEFAULT>)
PROJECTIONS QUERY EXPIRY: 5 (<DEFAULT>)
FAULT OUT OF ORDER PROJECTIONS: False (<DEFAULT>)
INT HTTP PREFIXES: <empty> (<DEFAULT>)
EXT HTTP PREFIXES: <empty> (<DEFAULT>)
ENABLE TRUSTED AUTH: False (<DEFAULT>)
ADD INTERFACE PREFIXES: True (<DEFAULT>)
CERTIFICATE STORE LOCATION: <empty> (<DEFAULT>)
CERTIFICATE STORE NAME: <empty> (<DEFAULT>)
CERTIFICATE SUBJECT NAME: <empty> (<DEFAULT>)
CERTIFICATE THUMBPRINT: <empty> (<DEFAULT>)
CERTIFICATE FILE: <empty> (<DEFAULT>)
CERTIFICATE PASSWORD: <empty> (<DEFAULT>)
USE INTERNAL SSL: False (<DEFAULT>)
DISABLE INSECURE TCP: False (<DEFAULT>)
SSL TARGET HOST: n/a (<DEFAULT>)
SSL VALIDATE SERVER: True (<DEFAULT>)
AUTHENTICATION TYPE: internal (<DEFAULT>)
AUTHENTICATION CONFIG: <empty> (<DEFAULT>)
DISABLE FIRST LEVEL HTTP AUTHORIZATION: False (<DEFAULT>)
PREPARE TIMEOUT MS: 2000 (<DEFAULT>)
COMMIT TIMEOUT MS: 2000 (<DEFAULT>)
UNSAFE DISABLE FLUSH TO DISK: False (<DEFAULT>)
BETTER ORDERING: False (<DEFAULT>)
UNSAFE IGNORE HARD DELETE: False (<DEFAULT>)
SKIP INDEX VERIFY: False (<DEFAULT>)
INDEX CACHE DEPTH: 16 (<DEFAULT>)
OPTIMIZE INDEX MERGE: False (<DEFAULT>)
GOSSIP INTERVAL MS: 1000 (<DEFAULT>)
GOSSIP ALLOWED DIFFERENCE MS: 60000 (<DEFAULT>)
GOSSIP TIMEOUT MS: 500 (<DEFAULT>)
ENABLE HISTOGRAMS: False (<DEFAULT>)
LOG HTTP REQUESTS: False (<DEFAULT>)
LOG FAILED AUTHENTICATION ATTEMPTS: False (<DEFAULT>)
ALWAYS KEEP SCAVENGED: False (<DEFAULT>)
SKIP INDEX SCAN ON READS: False (<DEFAULT>)
REDUCE FILE CACHE PRESSURE: False (<DEFAULT>)
INITIALIZATION THREADS: 1 (<DEFAULT>)
STRUCTURED LOG: True (<DEFAULT>)
MAX AUTO MERGE INDEX LEVEL: 2147483647 (<DEFAULT>)
WRITE STATS TO DB: True (<DEFAULT>)
[23028,01,23:11:45.854] {"defaults":{"Help":"False","Version":"False","Log":"C:\\webstack\\event-store-5\\logs","Config":"","Defines":"System.String[]","WhatIf":"False","StartStandardProjections":"False","DisableHTTPCaching":"False","MonoMinThreadpoolSize":"10","IntIp":"127.0.0.1","ExtIp":"127.0.0.1","IntTcpPort":"1112","IntSecureTcpPort":"0","ExtTcpPort":"1113","ExtSecureTcpPortAdvertiseAs":"0","ExtSecureTcpPort":"0","ExtIpAdvertiseAs":null,"ExtTcpPortAdvertiseAs":"0","ExtHttpPortAdvertiseAs":"0","IntIpAdvertiseAs":null,"IntSecureTcpPortAdvertiseAs":"0","IntTcpPortAdvertiseAs":"0","IntHttpPortAdvertiseAs":"0","IntTcpHeartbeatTimeout":"700","ExtTcpHeartbeatTimeout":"1000","IntTcpHeartbeatInterval":"700","ExtTcpHeartbeatInterval":"2000","GossipOnSingleNode":"False","ConnectionPendingSendBytesThreshold":"10485760","ConnectionQueueSizeThreshold":"50000","Force":"False","ClusterSize":"1","NodePriority":"0","MinFlushDelayMs":"2","CommitCount":"-1","PrepareCount":"-1","AdminOnExt":"True","StatsOnExt":"True","GossipOnExt":"True","DisableScavengeMerging":"False","ScavengeHistoryMaxAge":"30","DiscoverViaDns":"True","ClusterDns":"fake.dns","ClusterGossipPort":"30777","GossipSeed":"System.Net.IPEndPoint[]","StatsPeriodSec":"30","CachedChunks":"-1","ReaderThreadsCount":"4","ChunksCacheSize":"536871424","MaxMemTableSize":"1000000","HashCollisionReadLimit":"100","Db":"C:\\webstack\\event-store-5\\data","Index":null,"MemDb":"False","SkipDbVerify":"False","WriteThrough":"False","Unbuffered":"False","ChunkInitialReaderCount":"5","RunProjections":"None","ProjectionThreads":"3","WorkerThreads":"5","ProjectionsQueryExpiry":"5","FaultOutOfOrderProjections":"False","IntHttpPrefixes":"System.String[]","ExtHttpPrefixes":"System.String[]","EnableTrustedAuth":"False","AddInterfacePrefixes":"True","CertificateStoreLocation":"","CertificateStoreName":"","CertificateSubjectName":"","CertificateThumbprint":"","CertificateFile":"","CertificatePassword":"","UseInternalSsl":"False","DisableInsecureTCP":"False","SslTargetHost":"n\/a","SslValidateServer":"True","AuthenticationType":"internal","AuthenticationConfig":"","DisableFirstLevelHttpAuthorization":"False","PrepareTimeoutMs":"2000","CommitTimeoutMs":"2000","UnsafeDisableFlushToDisk":"False","BetterOrdering":"False","UnsafeIgnoreHardDelete":"False","SkipIndexVerify":"False","IndexCacheDepth":"16","OptimizeIndexMerge":"False","GossipIntervalMs":"1000","GossipAllowedDifferenceMs":"60000","GossipTimeoutMs":"500","EnableHistograms":"False","LogHttpRequests":"False","LogFailedAuthenticationAttempts":"False","AlwaysKeepScavenged":"False","SkipIndexScanOnReads":"False","ReduceFileCachePressure":"False","InitializationThreads":"1","StructuredLog":"True","MaxAutoMergeIndexLevel":"2147483647","WriteStatsToDb":"True"},"modified":{"ExtHttpPort":"2113","IntHttpPort":"2112"}}
[23028,01,23:11:45.864] Quorum size set to 1
[23028,01,23:11:45.864] Cannot find plugins path: "C:\webstack\event-store-5\plugins"
[23028,01,23:11:45.955] "INSTANCE ID:" 1d97c1fb-2f52-4bc7-9122-49318f3588c3
[23028,01,23:11:45.955] "DATABASE:" "C:\webstack\event-store-5\data"
[23028,01,23:11:45.955] "WRITER CHECKPOINT:" 96990866 (0x5C7F692)
[23028,01,23:11:45.962] "CHASER CHECKPOINT:" 96990866 (0x5C7F692)
[23028,01,23:11:45.962] "EPOCH CHECKPOINT:" 0 (0x0)
[23028,01,23:11:45.962] "TRUNCATE CHECKPOINT:" -1 (0xFFFFFFFFFFFFFFFF)
[23028,01,23:11:46.012] MessageHierarchy initialization took 00:00:00.0271714.
[23028,01,23:11:46.291] Opened ongoing "C:\webstack\event-store-5\data\chunk-000000.000000" as version 3
[23028,01,23:11:46.365] CACHED TFChunk #0-0 (chunk-000000.000000) in 00:00:00.0679940.
[23028,01,23:11:46.584] Starting MiniWeb for "/web/es/js/projections" ==> "C:\webstack\event-store-5\projections"
[23028,01,23:11:46.584] Starting MiniWeb for "/web/es/js/projections/v8/Prelude" ==> "C:\webstack\event-store-5\Prelude"
[23028,01,23:11:46.584] Starting MiniWeb for "/web" ==> "C:\webstack\event-store-5\clusternode-web"
[23028,01,23:11:46.584] Binding MiniWeb to "/web/{*remaining_path}"
[23028,20,23:11:46.599] ========== [127.0.0.1:2112] SYSTEM INIT...
[23028,20,23:11:46.599] Starting "Normal" TCP listening on TCP endpoint: 127.0.0.1:1113.
[23028,20,23:11:46.599] Starting HTTP server on ["http://127.0.0.1:2113/,http://localhost:2113/"]...
[23028,21,23:11:46.599] TableIndex initialization...
[23028,21,23:11:46.614] ReadIndex building...
[23028,20,23:11:46.599] Failed to start http server
EXCEPTION OCCURRED
Cannot access a disposed object.
Object name: 'System.Net.HttpListener'.
Exiting with exit code: 1.
Exit reason: HTTP async server failed to start listening at [http://127.0.0.1:2113/, http://localhost:2113/].
[23028,20,23:11:46.633] Exiting with exit code: 1.
Exit reason: "HTTP async server failed to start listening at [http://127.0.0.1:2113/, http://localhost:2113/]."
````
The ports are not in use:
C:\webstack\event-store-5> Get-Process -Id (Get-NetTCPConnection -LocalPort 2112).OwningProcess
Get-NetTCPConnection : No MSFT_NetTCPConnection objects found with property 'LocalPort' equal to '2112'. Verify the value of the property and retry.
C:\webstack\event-store-5> Get-Process -Id (Get-NetTCPConnection -LocalPort 2113).OwningProcess
Get-NetTCPConnection : No MSFT_NetTCPConnection objects found with property 'LocalPort' equal to '2113'. Verify the value of the property and retry.
```
EventStore details
- EventStore server version: 5.0.4.0
- Operating system: Windows 10 Version 10.0.17763 Build 17763
Additional context
This problem started suddenly and I have no idea from where it comes from. This might be a Windows problem as well. I've deleted the store, downloaded it and unpacked it, so I've tried this even with a fresh copy.
burzum
All 12 comments
burzum,
Might have the solution for you:
If you are trying to run Eventstore on 127.0.0.1 and it won't start then it maybe because windows has reserved port 2113 for one or more uris.
To determine if this is the case run the following command
netsh http show urlacl url=http://+:2113/
In the output check for any urls with the port 2113 and if there are any use the following command(s) to clear them.
netsh http delete urlacl url=http://+:2113/
netsh http delete urlacl url=http://127.0.0.1:2113/
netsh http delete urlacl url=http://localhost:2113/
netsh http delete urlacl url=http://
For local setup, this is a suggested setup:
Open up a new console window under Administrator:
netsh http add urlacl url=http://127.0.0.1:2113/ user="
StevenBlair123
on 14 Nov 2019
@StevenBlair123 thank you but it doesn't look like this is the cause.
What I don't understand is that it complains that the file already exists at the end of the output?
C:\WINDOWS\system32> netsh http show urlacl url=http://+:2113/
URL Reservations:
-----------------
C:\WINDOWS\system32> netsh http delete urlacl url=http://+:2113/
URL reservation delete failed, Error: 2
The system cannot find the file specified.
C:\WINDOWS\system32> whoami
asgard\florian
C:\WINDOWS\system32> netsh http add urlacl url=http://127.0.0.1:2113/ user="asgard\florian"
Url reservation add failed, Error: 183
Cannot create a file when that file already exists.
C:\WINDOWS\system32>
Hmm yeah I see.
So it does suggest there is a registration there (even though it is not showing it).
Funnily enough, my local event store is doing exactly the same as yours...
StevenBlair123
on 14 Nov 2019
I had one registration, removed it and added it again (running cmd under Admin)
The Event Store seems to start, but when i tried to hit the UI I get:
Service Unavailable
HTTP Error 503. The service is unavailable.
StevenBlair123
on 14 Nov 2019
@StevenBlair123 did you run any update recently? I've done windows updates as usual and I've installed Docker, after that the event store stopped working on this port I think. Docker also wanted me to enable the virtualization extension. Maybe there is a weird side effect of something?
I've also checked and removed the event store and added it back to the windows firewall settings.
burzum
on 14 Nov 2019
Yeah, I think you might be onto something.
I recall when this originally happened, it was after a Windows update.
The problem is trying to identifying what is causing this now.
StevenBlair123
on 14 Nov 2019
burzum,
If I run all 4 delete commands:
_netsh http delete urlacl url=http://+:2113/
netsh http delete urlacl url=http://127.0.0.1:2113/
netsh http delete urlacl url=http://localhost:2113/
netsh http delete urlacl url=http://you external ip:2113/_
Followed by these commands:
_netsh http add urlacl url=http://127.0.0.1:2113/ user="yourdomain\
netsh http add urlacl url=http://localhost:2113/ user="yourdomain\
netsh http add urlacl url=http://your external ip:2113/ user="yourdomain\
The EventStore now starts (but still getting the Gateway error for the UI)
StevenBlair123
on 14 Nov 2019
Running the first four commands resulted in this:
C:\WINDOWS\system32> netsh http delete urlacl url=http://+:2113/
URL reservation delete failed, Error: 2
The system cannot find the file specified.
C:\WINDOWS\system32> netsh http delete urlacl url=http://127.0.0.1:2113/
URL reservation successfully deleted
C:\WINDOWS\system32> netsh http delete urlacl url=http://localhost:2113/
URL reservation delete failed, Error: 2
The system cannot find the file specified.
C:\WINDOWS\system32> netsh http delete urlacl url=http://192.168.178.21:2113/
URL reservation successfully deleted
The next:
C:\WINDOWS\system32> netsh http add urlacl url=http://127.0.0.1:2113/ user="asgard\florian"
URL reservation successfully added
C:\WINDOWS\system32> netsh http add urlacl url=http://localhost:2113/ user="asgard\florian"
URL reservation successfully added
C:\WINDOWS\system32> netsh http add urlacl url=http://192.168.178.21:2113/ user="asgard\florian"
URL reservation successfully added
Attempt to start the event store:
[16172,20,10:20:40.317] Starting HTTP server on ["http://127.0.0.1:2113/,http://localhost:2113/"]...
[16172,21,10:20:40.317] ReadIndex building...
Exiting with exit code: 1.
Exit reason: HTTP async server failed to start listening at [http://127.0.0.1:2113/, http://localhost:2113/].
[16172,20,10:20:40.317] Failed to start http server
EXCEPTION OCCURRED
Cannot access a disposed object.
Object name: 'System.Net.HttpListener'.
[16172,20,10:20:40.348] Exiting with exit code: 1.
Exit reason: "HTTP async server failed to start listening at [http://127.0.0.1:2113/, http://localhost:2113/]."
Yeah, looks like you followed exactly as I did.
Might have tried these other things already:
- telnet 127.0.0.1 2113
- Launch event store from an administrator cmd window
3.Trying another user on the PC - Launching the Event Store on the External IP:
"EventStore.ClusterNode.exe" --db "db" --log "logs" --run-projections=all --start-standard-projections=true --skip-db-verify StatsPeriodSec --stats-period-sec=-1 --ext-ip=192.168.178.21
StevenBlair123
on 14 Nov 2019
C:\WINDOWS\system32> telnet 127.0.0.1 2113
Connecting To 127.0.0.1...Could not open connection to the host, on port 2113: Connect failed
C:\WINDOWS\system32> telnet 127.0.0.1 2112
Connecting To 127.0.0.1...Could not open connection to the host, on port 2112: Connect failed
C:\WINDOWS\system32>
Tried already running it as admin: No success either.
[29316,20,10:31:12.942] ========== [127.0.0.1:2112] SYSTEM INIT...
[29316,20,10:31:12.960] Starting "Normal" TCP listening on TCP endpoint: 192.168.178.21:1113.
[29316,20,10:31:12.960] Starting HTTP server on ["http://192.168.178.21:2113/"]...
[29316,21,10:31:12.960] TableIndex initialization...
[29316,21,10:31:12.960] ReadIndex building...
Exiting with exit code: 1.
Exit reason: HTTP async server failed to start listening at [http://192.168.178.21:2113/].
[29316,20,10:31:12.960] Failed to start http server
EXCEPTION OCCURRED
Cannot access a disposed object.
Object name: 'System.Net.HttpListener'.
[29316,20,10:31:12.989] Exiting with exit code: 1.
Exit reason: "HTTP async server failed to start listening at [http://192.168.178.21:2113/]."
Running it on another port works perfectly fine.
EventStore.ClusterNode.exe --run-projections=all --start-standard-projections=true --connection-pending-send-bytes-threshold=50485760 --ext-http-port 8012 --int-http-port 8013
Out of ideas sorry :(
StevenBlair123
on 14 Nov 2019
@StevenBlair123 thank you, no problem!
burzum
on 14 Nov 2019
Related issues
Euphoric
路
6Comments
haf
路
7Comments
andrii-litvinov
路
4Comments
ShawnMcCool
路
11Comments
pkruithof
路
5Comments